An Engine-Agnostic Deep Learning Framework in Java

Overview

DeepJavaLibrary

Continuous Continuous PyTorch Continuous Tensorflow Docs Nightly Publish

Deep Java Library (DJL)

Overview

Deep Java Library (DJL) is an open-source, high-level, engine-agnostic Java framework for deep learning. DJL is designed to be easy to get started with and simple to use for Java developers. DJL provides a native Java development experience and functions like any other regular Java library.

You don't have to be machine learning/deep learning expert to get started. You can use your existing Java expertise as an on-ramp to learn and use machine learning and deep learning. You can use your favorite IDE to build, train, and deploy your models. DJL makes it easy to integrate these models with your Java applications.

Because DJL is deep learning engine agnostic, you don't have to make a choice between engines when creating your projects. You can switch engines at any point. To ensure the best performance, DJL also provides automatic CPU/GPU choice based on hardware configuration.

DJL's ergonomic API interface is designed to guide you with best practices to accomplish deep learning tasks. The following pseudocode demonstrates running inference:

    // Assume user uses a pre-trained model from model zoo, they just need to load it
    Criteria<Image, Classifications> criteria =
            Criteria.builder()
                    .optApplication(Application.CV.OBJECT_DETECTION) // find object dection model
                    .setTypes(Image.class, Classifications.class) // define input and output
                    .optFilter("backbone", "resnet50") // choose network architecture
                    .build();

    try (ZooModel<Image, Classifications> model = ModelZoo.loadModel(criteria)) {
        try (Predictor<Image, Classifications> predictor = model.newPredictor()) {
            Image img = ImageFactory.getInstance().fromUrl("http://..."); // read image
            Classifications result = predictor.predict(img);

            // get the classification and probability
            ...
        }
    }

The following pseudocode demonstrates running training:

    // Construct your neural network with built-in blocks
    Block block = new Mlp(28, 28);

    try (Model model = Model.newInstance("mlp")) { // Create an empty model
        model.setBlock(block); // set neural network to model

        // Get training and validation dataset (MNIST dataset)
        Dataset trainingSet = new Mnist.Builder().setUsage(Usage.TRAIN) ... .build();
        Dataset validateSet = new Mnist.Builder().setUsage(Usage.TEST) ... .build();

        // Setup training configurations, such as Initializer, Optimizer, Loss ...
        TrainingConfig config = setupTrainingConfig();
        try (Trainer trainer = model.newTrainer(config)) {
            /*
             * Configure input shape based on dataset to initialize the trainer.
             * 1st axis is batch axis, we can use 1 for initialization.
             * MNIST is 28x28 grayscale image and pre processed into 28 * 28 NDArray.
             */
            Shape inputShape = new Shape(1, 28 * 28);
            trainer.initialize(new Shape[] {inputShape});

            EasyTrain.fit(trainer, epoch, trainingSet, validateSet);
        }

        // Save the model
        model.save(modelDir, "mlp");
    }

Getting Started

Resources

Release Notes

Building From Source

To build from source, begin by checking out the code. Once you have checked out the code locally, you can build it as follows using Gradle:

# for Linux/macOS:
./gradlew build

# for Windows:
gradlew build

To increase build speed, you can use the following command to skip unit tests:

# for Linux/macOS:
./gradlew build -x test

# for Windows:
gradlew build -x test

Note: SpotBugs is not compatible with JDK 11+. SpotBugs will not be executed if you are using JDK 11+.

Importing into eclipse

to import source project into eclipse

# for Linux/macOS:
./gradlew eclipse


# for Windows:
gradlew eclipse

in eclipse

file->import->gradle->existing gradle project

Note: please set your workspace text encoding setting to UTF-8

Community

You can read our guide to community forums, following DJL, issues, discussions, and RFCs to figure out the best way to share and find content from the DJL community.

Join our slack channel to get in touch with the development team, for questions and discussions.

Follow our twitter to see updates about new content, features, and releases.

关注我们 知乎专栏 获取DJL最新的内容!

Useful Links

License

This project is licensed under the Apache-2.0 License.

Comments
  • Possible memory-leak in multi-threading inference using Tensorflow and having org.bytedeco.javacpp.nopointergc=true (CPU)

    Possible memory-leak in multi-threading inference using Tensorflow and having org.bytedeco.javacpp.nopointergc=true (CPU)

    Description

    Possible memory-leak in multi-threading inference using Tensorflow and having org.bytedeco.javacpp.nopointergc=true

    CPU inference.

    Expected Behavior

    Garbage collection removing objects from Old Generation

    Error Message

    Java OOM

    How to Reproduce?

    Run multi-threaded inference for 30minutes with -Dorg.bytedeco.javacpp.nopointergc=true so you don't have JavaCPP Deallocator blocking thread.

    /gradlew benchmark -Dorg.bytedeco.javacpp.nopointergc=true -Dai.djl.default_engine=TensorFlow -Dai.djl.repository.zoo.location="https://storage.googleapis.com/tfhub-modules/tensorflow/resnet_50/classification/1.tar.gz?artifact_id=tf_resnet" --args='-n tf_resnet -t 10 -c 1000000 -s 1,224,224,3'

    bug Tensorflow 
    opened by skirdey 56
  • Request for help.

    Request for help.

    Information

    Hey there!

    I wanted to ask for help since I couldn't figure out how to fix a certain bug, and because I did not get an answer in 5 days on the DJL Slack channel.

    Related Slack Related Exception

    Issue

    The Issue that I am having is that the BinaryImageTranslator I created has Issue with any Image given to it, I could not figure out why exactly but as of it looks there is an Issue with the resizing/reshaping of the Input Image.

    opened by DxsSucuk 51
  • MXNET GPU CUDA missmatch Problem?

    MXNET GPU CUDA missmatch Problem?

    hey i m new here ,today i have successfully install ijava kernel in google colab and java is running successfully BUT when i train i got this ERROR : ai.djl.engine.EngineException: MXNet engine call failed: MXNetError: Compile with USE_CUDA=1 to enable GPU usage Stack trace: File "src/storage/storage.cc", line 119 at ai.djl.mxnet.jna.JnaUtils.checkCall(JnaUtils.java:1909) at ai.djl.mxnet.jna.JnaUtils.createNdArray(JnaUtils.java:349) at ai.djl.mxnet.engine.MxNDManager.create(MxNDManager.java:91) at ai.djl.mxnet.engine.MxNDManager.create(MxNDManager.java:34) at ai.djl.ndarray.NDManager.create(NDManager.java:526) at ai.djl.mxnet.engine.MxNDArray.duplicate(MxNDArray.java:184) at ai.djl.mxnet.engine.MxNDArray.toDevice(MxNDArray.java:197) at ai.djl.training.ParameterStore.getValue(ParameterStore.java:110) at ai.djl.training.Trainer.lambda$initialize$1(Trainer.java:120) at java.base/java.lang.Iterable.forEach(Iterable.java:75) at ai.djl.training.Trainer.initialize(Trainer.java:117) at .(#76:1) heres my training phase:

    int batchSize = 32; int limit = Integer.MAX_VALUE; // change this to a small value for a dry run // int limit = 160; // limit 160 records in the dataset for a dry run Pipeline pipeline = new Pipeline( new ToTensor(), new Normalize(new float[] {0.4914f, 0.4822f, 0.4465f}, new float[] {0.2023f, 0.1994f, 0.2010f})); Cifar10 trainDataset = Cifar10.builder() .setSampling(batchSize, true) .optUsage(Dataset.Usage.TRAIN) .optLimit(limit) .optPipeline(pipeline) .build(); trainDataset.prepare(new ProgressBar()); DefaultTrainingConfig config = new DefaultTrainingConfig(Loss.softmaxCrossEntropyLoss()) //softmaxCrossEntropyLoss is a standard loss for classification problems .addEvaluator(new Accuracy()) // Use accuracy so we humans can understand how accurate the model is .optDevices(new Device[]{Device.gpu(0)}) // Limit your GPU, using more GPU actually will slow down coverging .addTrainingListeners(TrainingListener.Defaults.logging()); // Now that we have our training configuration, we should create a new trainer for our model Trainer trainer = model.newTrainer(config); int epoch = 10; Shape inputShape = new Shape(1, 3, 32, 32); trainer.initialize(inputShape); for (int i = 0; i < epoch; ++i) { int index = 0; for (Batch batch : trainer.iterateDataset(trainDataset)) { EasyTrain.trainBatch(trainer, batch); trainer.step(); batch.close(); } // reset training and validation evaluators at end of epoch trainer.notifyListeners(listener -> listener.onEpoch(trainer)); }`

    I know it CUDA related error in google colab its showing cuda-10.0 installed i have alse tried installing mxnet-cu90 using this Cmd: !pip install mxnet-cu90 Still not working .. Please help me through this ??

    bug 
    opened by nikkisingh111333 28
  • EngineException when running in GraalVM Native Image

    EngineException when running in GraalVM Native Image

    Description

    Please see this updated project: https://github.com/murphye/djl-demo/blob/master/pneumonia-detection-quarkus/README.md

    I have two endpoints:

    /detect will attempt to run the model, and will fail with the following error:

    Caused by: ai.djl.engine.EngineException: No deep learning engine found.
    Please refer to https://github.com/awslabs/djl/blob/master/docs/development/troubleshooting.md for more details.
            at ai.djl.engine.Engine.getInstance(Engine.java:90)
            at ai.djl.repository.zoo.Criteria$Builder.<init>(Criteria.java:193)
            at ai.djl.repository.zoo.Criteria.builder(Criteria.java:173)
            at com.example.ExampleResource.detect(ExampleResource.java:45)
    

    /check will print out information showing that the TensorFlow Engine has been loaded properly.

    >>>>>>>> ZooProvider: ai.djl.repository.zoo.DefaultZooProvider [models/saved_model:ai.djl.localmodelzoo:saved_model [
            ai.djl.localmodelzoo:saved_model:N/A {}
    
    ]]
    >>>>>>>> Engine: TensorFlow
    

    There is a contradiction of information here as TensorFlow Engine is loading, but still doesn't work. This only occurs when running in Native mode, as JVM mode is OK

    I am not sure how to proceed. I suggest trying to add more debugging info to DJL to understand what is happening. Quarkus team will have debugger support soon.

    How to Reproduce?

    See instructions https://github.com/murphye/djl-demo/blob/master/pneumonia-detection-quarkus/README.md

    Run the two endpoints and observe the logs.

    What have you tried to solve it?

    These configs enable the ServiceLoader mechanism:

    1. https://github.com/murphye/djl-demo/blob/master/pneumonia-detection-quarkus/src/main/resources/reflection-config.json
    2. https://github.com/murphye/djl-demo/blob/master/pneumonia-detection-quarkus/src/main/resources/resources-config.json

    Also see https://github.com/murphye/djl-demo/blob/master/pneumonia-detection-quarkus/src/main/resources/application.properties

    Environment Info

    Using Maven rather than Gradle as it provides error messages that Gradle does not for Native compilation. Using Mac and GraalVM 20.1.0 with Quarkus 1.5.1.Final

    bug 
    opened by murphye 28
  • Inference tensors cannot be saved for backward.

    Inference tensors cannot be saved for backward.

    I don't know if this is a bug. I'm trying to follow the official tutorial using pytorch engine.

    Here is my code and exception.

    String modelUrl = "https://resources.djl.ai/test-models/traced_distilbert_wikipedia_uncased.zip";
    Criteria<NDList, NDList> criteria = Criteria.builder()
        .optApplication(Application.NLP.WORD_EMBEDDING)
        .setTypes(NDList.class, NDList.class)
        .optModelUrls(modelUrl)
        .optProgress(new ProgressBar())
        .build();
    
    ZooModel<NDList, NDList> embedding = criteria.loadModel();
    Predictor<NDList, NDList> embedder = embedding.newPredictor();
    SequentialBlock classifier = new SequentialBlock()
        .add(
                ndList -> {
                    NDArray data = ndList.singletonOrThrow();
                    long batchSize = data.getShape().get(0);
                    long maxLen = data.getShape().get(1);
                    NDList inputs = new NDList();
                    inputs.add(data.toType(DataType.INT64, false));
                    inputs.add(data.getManager().full(data.getShape(), 1, DataType.INT64));
                    inputs.add(data.getManager().arange(maxLen).toType(DataType.INT64, false).broadcast(data.getShape()));
                    try {
                        return embedder.predict(inputs);
                    } catch (TranslateException e) {
                        throw new RuntimeException(e);
                    }
                }
        )
        .add(Linear.builder().setUnits(768).build())
        .add(Activation::relu)
        .add(Dropout.builder().optRate(0.2f).build())
        .add(Linear.builder().setUnits(5).build())
        .addSingleton(nd -> nd.get(":,0"));
    
    Model model = Model.newInstance("review_classification");
    model.setBlock(classifier);
    
    DefaultVocabulary vocabulary = DefaultVocabulary.builder()
        .addFromTextFile(embedding.getArtifact("vocab.txt"))
        .optUnknownToken("[UNK]")
        .build();
    
    int maxTokenLen = 64;
    int batchSize = 8;
    int limit = Integer.MAX_VALUE;
    
    BertFullTokenizer tokenizer = new BertFullTokenizer(vocabulary, true);
    CsvDataset awsDataset = getDataset(batchSize, tokenizer, maxTokenLen, limit);
    RandomAccessDataset[] datasets = awsDataset.randomSplit(7, 3);
    RandomAccessDataset trainDataset = datasets[0];
    RandomAccessDataset evalDataset = datasets[1];
    
    SaveModelTrainingListener listener = new SaveModelTrainingListener("build/model");
    listener.setSaveModelCallback(
        trainer -> {
            TrainingResult result = trainer.getTrainingResult();
            Model trainerModel = trainer.getModel();
            float acc = result.getValidateEvaluation("Accuracy");
            trainerModel.setProperty("Accuracy", String.format("%.5f", acc));
            trainerModel.setProperty("Loss", String.format("%.5f", result.getValidateLoss()));
        }
    );
    
    DefaultTrainingConfig trainingConfig = new DefaultTrainingConfig(Loss.softmaxCrossEntropyLoss())
        .addEvaluator(new Accuracy())
        .addTrainingListeners(TrainingListener.Defaults.logging("build/model"))
        .addTrainingListeners(listener);
    
    int epoch = 2;
    Trainer trainer = model.newTrainer(trainingConfig);
    trainer.setMetrics(new Metrics());
    Shape shape = new Shape(batchSize, maxTokenLen);
    trainer.initialize(shape);
    EasyTrain.fit(trainer, epoch, trainDataset, evalDataset);
    System.out.println(trainer.getTrainingResult());
    model.save(Paths.get("build/model"), "aws-review-rank");
    
    [main] INFO ai.djl.pytorch.engine.PtEngine - Number of inter-op threads is 6
    [main] INFO ai.djl.pytorch.engine.PtEngine - Number of intra-op threads is 6
    [main] INFO ai.djl.training.listener.LoggingTrainingListener - Training on: cpu().
    [main] INFO ai.djl.training.listener.LoggingTrainingListener - Load PyTorch Engine Version 1.12.1 in 0.079 ms.
    Exception in thread "main" ai.djl.engine.EngineException: Inference tensors cannot be saved for backward. To work around you can make a clone to get a normal tensor and use it in autograd.
    	at ai.djl.pytorch.jni.PyTorchLibrary.torchNNLinear(Native Method)
    	at ai.djl.pytorch.jni.JniUtils.linear(JniUtils.java:1189)
    	at ai.djl.pytorch.engine.PtNDArrayEx.linear(PtNDArrayEx.java:390)
    	at ai.djl.nn.core.Linear.linear(Linear.java:183)
    	at ai.djl.nn.core.Linear.forwardInternal(Linear.java:88)
    	at ai.djl.nn.AbstractBaseBlock.forwardInternal(AbstractBaseBlock.java:126)
    	at ai.djl.nn.AbstractBaseBlock.forward(AbstractBaseBlock.java:91)
    	at ai.djl.nn.SequentialBlock.forwardInternal(SequentialBlock.java:209)
    	at ai.djl.nn.AbstractBaseBlock.forward(AbstractBaseBlock.java:91)
    	at ai.djl.training.Trainer.forward(Trainer.java:175)
    	at ai.djl.training.EasyTrain.trainSplit(EasyTrain.java:122)
    	at ai.djl.training.EasyTrain.trainBatch(EasyTrain.java:110)
    	at ai.djl.training.EasyTrain.fit(EasyTrain.java:58)
    	at cn.amberdata.misc.djl.rankcls.Main.main(Main.java:114)
    

    And here is my dependencies.

            <!-- https://mvnrepository.com/artifact/ai.djl/api -->
            <dependency>
                <groupId>ai.djl</groupId>
                <artifactId>api</artifactId>
                <version>0.19.0</version>
            </dependency>
            <!-- https://mvnrepository.com/artifact/org.slf4j/slf4j-simple -->
            <dependency>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-simple</artifactId>
                <version>1.7.36</version>
            </dependency>
            <!-- https://mvnrepository.com/artifact/ai.djl.pytorch/pytorch-engine -->
            <dependency>
                <groupId>ai.djl.pytorch</groupId>
                <artifactId>pytorch-engine</artifactId>
                <version>0.19.0</version>
            </dependency>
            <!-- https://mvnrepository.com/artifact/ai.djl/basicdataset -->
            <dependency>
                <groupId>ai.djl</groupId>
                <artifactId>basicdataset</artifactId>
                <version>0.19.0</version>
            </dependency>
            <!-- https://mvnrepository.com/artifact/ai.djl/model-zoo -->
            <dependency>
                <groupId>ai.djl</groupId>
                <artifactId>model-zoo</artifactId>
                <version>0.19.0</version>
            </dependency>
    
    opened by SuperMaskv 22
  • NullPointerException on trainer initializing

    NullPointerException on trainer initializing

    hey why i m getting error while setting shape with trainer and initializing it heres my code:

    **` Model model = Model.newInstance("pikachu-ssd"); float detectionThreshold = 0.6f; // load parameters back to original training block model.setBlock(ssd); // append prediction logic at end of training block with parameter loaded Block ssdTrain = model.getBlock(); model.setBlock(ssdPredict); var epoch=10; var train_config= new DefaultTrainingConfig(new SingleShotDetectionLoss()) .addEvaluator(new SingleShotDetectionAccuracy("classAccuracy")) .addEvaluator(new BoundingBoxError("boundingBoxError")) .optDevices(new Device[]{Device.gpu(0)}) .addTrainingListeners(TrainingListener.Defaults.logging()); Trainer trainer = model.newTrainer(train_config); SingleShotDetectionTranslator translator = SingleShotDetectionTranslator.builder() .addTransform(new ToTensor()) .optSynset(Collections.singletonList("pikachu")) .optThreshold(detectionThreshold) .build(); trainer.setMetrics(new Metrics()); System.out.println("whaaat"); trainer.initialize(new Shape(150, 3, 256, 256)); for (int i = 0; i < epoch; ++i) { int index = 0; for (var batch : trainer.iterateDataset(pikachuData)) { System.out.println("loo"); EasyTrain.trainBatch(trainer, batch); trainer.step(); batch.close(); } // reset training and validation evaluators at end of epoch trainer.notifyListeners(listener -> listener.onEpoch(trainer));

    }`**

    And I m Getting This : `** java.lang.NullPointerException: null

        at ai.djl.mxnet.engine.MxNDArray.createGradient(MxNDArray.java:244)
    
        at ai.djl.mxnet.engine.MxNDArray.attachGradient(MxNDArray.java:229)
    
    at ai.djl.nn.Parameter.initialize(Parameter.java:213)
    
    at ai.djl.nn.AbstractBlock.initialize(AbstractBlock.java:269)
    
    at ai.djl.nn.SequentialBlock.initializeChildBlocks(SequentialBlock.java:143)
    
    at ai.djl.nn.AbstractBlock.initialize(AbstractBlock.java:271)
    
    at ai.djl.nn.SequentialBlock.initializeChildBlocks(SequentialBlock.java:143)
    
    at ai.djl.nn.AbstractBlock.initialize(AbstractBlock.java:271)
    
    at ai.djl.basicmodelzoo.cv.object_detection.ssd.SingleShotDetection.initialize(SingleShotDetection.java:184)
    
    at ai.djl.nn.SequentialBlock.initializeChildBlocks(SequentialBlock.java:143)
    
    at ai.djl.nn.AbstractBlock.initialize(AbstractBlock.java:271)
    
    at ai.djl.training.Trainer.initialize(Trainer.java:113)
    
    at .(#249:1)**
    

    ` No clues whats pointing this error heres my maven and import tree take a look:

    %maven ai.djl:api:0.10.0 %maven ai.djl:model-zoo:0.10.0 %maven ai.djl.mxnet:mxnet-engine:0.10.0 %maven ai.djl.mxnet:mxnet-model-zoo:0.10.0 %maven org.slf4j:slf4j-api:1.7.26 %maven org.slf4j:slf4j-simple:1.7.26 %maven net.java.dev.jna:jna:5.3.0 %maven ai.djl:basicdataset:0.10.0 // See https://github.com/awslabs/djl/blob/master/mxnet/mxnet-engine/README.md // for more MXNet library selection options %maven ai.djl.mxnet:mxnet-native-cu101mkl:1.7.0-backport import ai.djl.Device; import ai.djl.MalformedModelException; import ai.djl.Model; import ai.djl.basicdataset.cv.PikachuDetection; import ai.djl.basicmodelzoo.cv.object_detection.ssd.SingleShotDetection; //import ai.djl.examples.training.util.Arguments; import ai.djl.inference.Predictor; import ai.djl.metric.Metrics; import ai.djl.modality.cv.Image; import ai.djl.modality.cv.ImageFactory; import ai.djl.modality.cv.MultiBoxDetection; import ai.djl.modality.cv.output.DetectedObjects; import ai.djl.modality.cv.transform.ToTensor; import ai.djl.modality.cv.translator.SingleShotDetectionTranslator; import ai.djl.ndarray.NDArray; import ai.djl.ndarray.NDList; import ai.djl.ndarray.types.Shape; import ai.djl.nn.Block; import ai.djl.nn.LambdaBlock; import ai.djl.nn.SequentialBlock; import ai.djl.training.DefaultTrainingConfig; import ai.djl.training.EasyTrain; import ai.djl.training.Trainer; import ai.djl.training.TrainingResult; import ai.djl.training.dataset.Dataset; import ai.djl.training.dataset.RandomAccessDataset; import ai.djl.training.evaluator.BoundingBoxError; import ai.djl.training.evaluator.SingleShotDetectionAccuracy; import ai.djl.training.listener.SaveModelTrainingListener; import ai.djl.training.listener.TrainingListener; import ai.djl.training.loss.SingleShotDetectionLoss; import ai.djl.training.util.ProgressBar; import ai.djl.translate.Pipeline; import ai.djl.translate.TranslateException; import java.io.IOException; import java.nio.file.Files; import java.nio.file.Path; import java.nio.file.Paths; import java.util.ArrayList; import java.util.Arrays; import java.util.Collections; import java.util.List;**

    help me out please ..

    bug 
    opened by nikkisingh111333 22
  • java.nio.ReadOnlyBufferException when calling TensorFlow model.

    java.nio.ReadOnlyBufferException when calling TensorFlow model.

    Question

    I am trying to call a Tensorflow model and keep getting a ReadOnlyBufferException. Originally this was an .hdf5 model which I converted to .pb file to use with djl as per instructions here.

    The model input (in python) is a float64 numpy array of shape (N, 40,40,1). The model loads fine using djl and I've created a translator which inputs a double[][] array of the same shape but when calling NDArray array = manager.create(specgramFlat, shape); I get a ReadOnlyBufferException error.

    A minimal reproducible example is below and you can download the zipped model https://1drv.ms/u/s!AkNvdu-1_rHOgahqrZwrhu6V8v3TFA?e=0BR4a3.

    Am I going about loading this model the right way? Any help on this would be much appreciated. Thanks!

    import java.io.File;
    import java.nio.file.Path;
    import java.nio.file.Paths;
    import java.util.LinkedList;
    import java.util.Random;
    
    import ai.djl.Model;
    import ai.djl.engine.Engine;
    import ai.djl.inference.Predictor;
    import ai.djl.ndarray.NDArray;
    import ai.djl.ndarray.NDList;
    import ai.djl.ndarray.NDManager;
    import ai.djl.ndarray.types.Shape;
    import ai.djl.translate.Batchifier;
    import ai.djl.translate.Translator;
    import ai.djl.translate.TranslatorContext;
    
    
    /**
     * A minimal reproducible example of a java.nio.ReadOnlyBufferException when trying to call a tensorflow classifier. 
     * 
     * @author Jamie Macaulay 
     */
    public class ReadBufferExceptionTest {
    
    
    	public static void main(String[] args) {
    
    		String modelPath = "saved_model.pb";
    
    		try {
    
    			//load the Tensorflow model. 
    			File file = new File(modelPath); 
    
    			Path modelDir = Paths.get(file.getAbsoluteFile().getParent()); //the directory of the file (in case the file is local this should also return absolute directory)
    
    			System.out.println(Engine.getAllEngines()); 
    
    			Model model = Model.newInstance(modelPath, "TensorFlow"); 
    
    			model.load(modelDir, "saved_model.pb");
    
    			System.out.println("Input: " + model.describeInput().values()); 
    			System.out.println("Output: " + model.describeOutput().values()); 
    
    			//create the predictor
    			Translator<double[][], float[]>  translator = new Translator<double[][], float[]>() {   
    				
    
    				@Override
    				public NDList processInput(TranslatorContext ctx, double[][] data) {
    					//System.out.println("Hello: 1 " ); 
    					NDManager manager = ctx.getNDManager();
    
    					Shape shape = new Shape(1L, data.length, data[0].length, 1L); 
    
    					System.out.println("NDArray shape: " + shape); 
    
    					double[] specgramFlat = flattenDoubleArray(data); 
    
    					NDArray array = manager.create(specgramFlat, shape); 
    					//		NDArray array = manager.create(data); 
    
    					System.out.println("NDArray size: " + array.size()); 
    
    					return new NDList (array);
    				}
    
    				@Override
    				public float[]  processOutput(TranslatorContext ctx, NDList list) {
    					System.out.println("Hello: 2 " + list); 
    
    					NDArray temp_arr = list.get(0);
    
    					Number[] number = temp_arr.toArray(); 
    
    					float[] results = new float[number.length]; 
    					for (int i=0; i<number.length; i++) {
    						results[i] = number[i].floatValue(); 
    					}
    
    					return results; 
    				}
    
    				@Override
    				public Batchifier getBatchifier() {
    					// The Batchifier describes how to combine a batch together
    					// Stacking, the most common batchifier, takes N [X1, X2, ...] arrays to a single [N, X1, X2, ...] array
    					return Batchifier.STACK;
    				}
    			};
    			Predictor<double[][], float[]> predictor = model.newPredictor(translator);
    			
    			
    			//make some fake data for input
    			double[][] data = makeDummySpectrogramd(40, 40); 
    
    			Shape shape = new Shape(1L, 40, 40, 1L); 
    
    			System.out.println("NDArray shape: " + shape); 
    			
    			//			NDArray array = manager.create(specgramFlat, shape); 
    			model.getNDManager().create(data); 
    
    			float[] output = predictor.predict(data); 
    
    		}
    		catch (Exception e) {
    			e.printStackTrace();
    		}
    	}
    
    	
    	
    	/**
    	 * Make a dummy spectrogram for testing. Filled with random values.  
    	 * @param len - the length of the spectrogram in bins. 
    	 * @param height - the height of the spectrgram in bins. 
    	 * @return a dummy spectrogram with random values. 
    	 */
    	public static double[][] makeDummySpectrogramd(int len, int len2){
    
    		//		int len = 256; 
    		//		int len2 = 128; 
    
    		double[][] specDummy = new double[len][len2]; 
    
    		Random rand = new Random(); 
    		for (int i=0; i<len; i++){
    			for (int j=0; j<len2; j++) {
    				specDummy[i][j] = 2F*(rand.nextFloat()-0.5F);
    
    				if (specDummy[i][j]>1) {
    					specDummy[i][j]=1F;
    				}
    				if (specDummy[i][j]<0) {
    					specDummy[i][j]=0F;
    				}
    			}
    		}
    		return specDummy; 
    	}
    	
    
    	/** 
    	 * Convert an arbitrary-dimensional rectangular double array to flat vector.<br>
    	 * Can pass double[], double[][], double[][][], etc.
    	 */
    	public static double[] flattenDoubleArray(Object doubleArray) {
    		if (doubleArray instanceof double[])
    			return (double[]) doubleArray;
    
    		LinkedList<Object> stack = new LinkedList<>();
    		stack.push(doubleArray);
    
    		int[] shape = arrayShape(doubleArray);
    		int length = prod(shape);
    		double[] flat = new double[length];
    		int count = 0;
    
    		while (!stack.isEmpty()) {
    			Object current = stack.pop();
    			if (current instanceof double[]) {
    				double[] arr = (double[]) current;
    				for (int i = 0; i < arr.length; i++)
    					flat[count++] = arr[i];
    			} else if (current instanceof Object[]) {
    				Object[] o = (Object[]) current;
    				for (int i = o.length - 1; i >= 0; i--)
    					stack.push(o[i]);
    			} else
    				throw new IllegalArgumentException("Base array is not double[]");
    		}
    
    		if (count != flat.length)
    			throw new IllegalArgumentException("Fewer elements than expected. Array is ragged?");
    		return flat;
    	}
    	
    	/** Calculate the shape of an arbitrary multi-dimensional array. Assumes:<br>
    	 * (a) array is rectangular (not ragged) and first elements (i.e., array[0][0][0]...) are non-null <br>
    	 * (b) First elements have > 0 length. So array[0].length > 0, array[0][0].length > 0, etc.<br>
    	 * Can pass any Java array opType: double[], Object[][][], float[][], etc.<br>
    	 * Length of returned array is number of dimensions; returned[i] is size of ith dimension.
    	 */
    	public static int[] arrayShape(Object array) {
    		int nDimensions = 0;
    		Class<?> c = array.getClass().getComponentType();
    		while (c != null) {
    			nDimensions++;
    			c = c.getComponentType();
    		}
    
    
    		int[] shape = new int[nDimensions];
    		Object current = array;
    		for (int i = 0; i < shape.length - 1; i++) {
    			shape[i] = ((Object[]) current).length;
    			current = ((Object[]) current)[0];
    		}
    
    		if (current instanceof Object[]) {
    			shape[shape.length - 1] = ((Object[]) current).length;
    		} else if (current instanceof double[]) {
    			shape[shape.length - 1] = ((double[]) current).length;
    		} else if (current instanceof float[]) {
    			shape[shape.length - 1] = ((float[]) current).length;
    		} else if (current instanceof long[]) {
    			shape[shape.length - 1] = ((long[]) current).length;
    		} else if (current instanceof int[]) {
    			shape[shape.length - 1] = ((int[]) current).length;
    		} else if (current instanceof byte[]) {
    			shape[shape.length - 1] = ((byte[]) current).length;
    		} else if (current instanceof char[]) {
    			shape[shape.length - 1] = ((char[]) current).length;
    		} else if (current instanceof boolean[]) {
    			shape[shape.length - 1] = ((boolean[]) current).length;
    		} else if (current instanceof short[]) {
    			shape[shape.length - 1] = ((short[]) current).length;
    		} else
    			throw new IllegalStateException("Unknown array opType"); //Should never happen
    		return shape;
    	}
    
    	
    
    	/**
    	 * Product of an int array
    	 * @param mult the elements
    	 *            to calculate the sum for
    	 * @return the product of this array
    	 */
    	public static int prod(int... mult) {
    		if (mult.length < 1)
    			return 0;
    		int ret = 1;
    		for (int i = 0; i < mult.length; i++)
    			ret *= mult[i];
    		return ret;
    	}
    
    
    	
    }
    
    
    question 
    opened by macster110 19
  • Can not load Mxnet trained model

    Can not load Mxnet trained model

    Description

    Can not load a pre trained Yolo model from Mxnet. I have a param file and a symbol.json. MxModel seems to fail to handle the params file. If interested I might be able to share the model private on request. The model was trained in a mxnet/gluoncv python environment.

    Debugging the code I can see the key value is stages.0.0.0.weight which is supposed to be split by ":" which obviously fails.

    Error Message

    Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1
    	at ai.djl.mxnet.engine.MxModel.loadParameters(MxModel.java:201)
    	at ai.djl.mxnet.engine.MxModel.load(MxModel.java:119)
    	at ai.djl.repository.zoo.BaseModelLoader.loadModel(BaseModelLoader.java:142)
    	at ai.djl.repository.zoo.ModelZoo.loadModel(ModelZoo.java:162)
    	at com.itth.okra.axle.AxleDetectorMxnet.(AxleDetectorMxnet.java:29)
    	at com.itth.okra.axle.AxleDetectorMxnet.main(AxleDetectorMxnet.java:42)
    

    How to Reproduce?

    I try to load the model with following code:

             Criteria criteria = Criteria.builder()
                   .setTypes(Image.class, DetectedObjects.class) // defines input and output data type
                   .optDevice(Device.cpu())
                   .optTranslator(new YoloTranslator(new Builder()))
                   .optModelUrls("file:///tmp/mxnet") // search models in specified path
                   .optModelName("model")
                   .build();
             final ZooModel model = ModelZoo.loadModel(criteria);
    

    Environment Info

    djl: 0.8.0 mxnet-engine: 0.8.0 mxnet-native-mkl: 1.7.0

    bug 
    opened by thhart 19
  • jvm crushes in native when try to running torch model

    jvm crushes in native when try to running torch model

    Description

    Hi. I have some problems with running a model from https://github.com/emiliantolo/pytorch_nsfw_model in DJL. Jvm crashes with an error in native. I try to run it with openjdk 8, zulu 8, zulu 13.

    Expected Behavior

    Expected that a model will run correctly

    Error Message

    #
    # A fatal error has been detected by the Java Runtime Environment:
    #
    #  SIGSEGV (0xb) at pc=0x0000000134a917ae, pid=12860, tid=9219
    #
    # JRE version: OpenJDK Runtime Environment 
    (Zulu13.28+11-CA) (13.0.1+10) (build 13.0.1+10-MTS)
    # Java VM: OpenJDK 64-Bit Server VM (13.0.1+10-MTS, mixed mode, sharing, tiered, compressed oops, g1 gc, bsd-amd64)
    # Problematic frame:
    # C  [libtorch_cpu.dylib+0x2a957ae]  torch::jit::Expr::Expr(c10::intrusive_ptr<torch::jit::Tree, c10::detail::intrusive_target_default_null_type<torch::jit::Tree> > const&)+0x2e
    #
    # No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
    #
    # An error report file with more information is saved as:
    # /Users/evgenyzakharov/Workspace/pytorch_nsfw_model_jvm/hs_err_pid12860.log
    #
    # If you would like to submit a bug report, please visit:
    #   http://www.azulsystems.com/support/
    # The crash happened outside the Java Virtual Machine in native code.
    # See problematic frame for where to report the bug.
    #
    

    And in dump error message is next:

    Stack: [0x000070000ea99000,0x000070000eb99000],  sp=0x000070000eb95f50,  free space=1011k
    Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code)
    C  [libtorch_cpu.dylib+0x2a957ae]  torch::jit::Expr::Expr(c10::intrusive_ptr<torch::jit::Tree, c10::detail::intrusive_target_default_null_type<torch::jit::Tree> > const&)+0x2e
    C  [libtorch_cpu.dylib+0x2d21cb5]  torch::jit::ScriptTypeParser::parseClassConstant(torch::jit::Assign const&)+0xd5
    C  [libtorch_cpu.dylib+0x2a9097a]  torch::jit::SourceImporterImpl::importClass(c10::QualifiedName const&, torch::jit::ClassDef const&, bool)+0x210a
    C  [libtorch_cpu.dylib+0x2a8c61a]  torch::jit::SourceImporterImpl::importNamedType(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, torch::jit::ClassDef const&)+0x64a
    C  [libtorch_cpu.dylib+0x2a88d2b]  torch::jit::SourceImporterImpl::findNamedType(c10::QualifiedName const&)+0xfb
    

    Full error log: hs_err_pid12860.log

    How to Reproduce?

    Download model from repository https://github.com/emiliantolo/pytorch_nsfw_model and try to run with DJL

    Steps to reproduce

    1. Download model
    2. Convert model to ".pt" with script or trace:
    model = models.resnet50()
    model.fc = nn.Sequential(nn.Linear(2048, 512),
                                     nn.ReLU(),
                                     nn.Dropout(0.2),
                                     nn.Linear(512, 10),
                                     nn.LogSoftmax(dim=1))
    model.load_state_dict(torch.load('ResNet50_nsfw_model.pth', map_location=torch.device('cpu')))
    model.eval()
    
    #script
    export = torch.jit.script(model)
    torch.jit.save(export, "out.pt")
    
    #trace
    image = Image.open(data_dir+"1.jpg")
    image_tensor = test_transforms(image).float()
    image_tensor = image_tensor.unsqueeze_(0)
    input = Variable(image_tensor)
    net_trace = torch.jit.trace(model, input)
    net_trace.save("out.pt")
    
    1. Try to use model in DJL with next criteria:
    val criteria = Criteria.builder()
            .setTypes(
                Image::class.java,
                Classifications::class.java
            )
            .optModelZoo(DefaultModelZoo("<path to folder with model>"))
            .optModelName("out.pt")
            .optTranslator(translator)
            .optProgress(ProgressBar())
            .build()
    

    Version in build.gradle.kts:

    
    implementation("ai.djl:api:0.6.0")
    runtimeOnly("ai.djl.pytorch:pytorch-engine:0.6.0")
    runtimeOnly("ai.djl.pytorch:pytorch-native-auto:1.5.0")
    

    What have you tried to solve it?

    1. Try to use different version of jvm (openjdk 8, zulu 8 zulu 13)
    2. Try to use different version of torch (1.6.0, 1.4.0)

    Environment Info

    OS:uname:Darwin 18.7.0 Darwin Kernel Version 18.7.0: Thu Jun 18 20:50:10 PDT 2020; root:xnu-4903.278.43~1/RELEASE_X86_64 x86_64
    rlimit: STACK 8192k, CORE 0k, NPROC 1418, NOFILE 10240, AS infinity, DATA infinity, FSIZE infinity
    load average:2.75 2.65 2.72
    
    CPU:total 8 (initial active 8) (4 cores per cpu, 2 threads per core) family 6 model 158 stepping 9, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, avx, avx2, aes, clmul, erms, 3dnowpref, lzcnt, ht, tsc, tscinvbit, bmi1, bmi2, adx, fma
    
    Memory: 4k page, physical 16777216k(62596k free), swap 6291456k(874240k free)
    
    vm_info: OpenJDK 64-Bit Server VM (13.0.1+10-MTS) for bsd-amd64 JRE (13.0.1+10-MTS) (Zulu13.28+11-CA), built on Oct  9 2019 12:07:25 by "zulu_re" with clang 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)
    
    bug 
    opened by evgzakharov 19
  • Allowing unloading the native libraries

    Allowing unloading the native libraries

    Description

    I am working on an application that allows switching between different pytorch versions dynamically. In order to do so I load dynamically the JARs needed fo each particular version on a child classloader of the main classloader. However I am not able to switch between versions because the child classloader is never garbage collected so the native library is never unloaded and loading a two native libraries even of different versions causes errors. Can you think of any workaround to tackle this issue.

    enhancement 
    opened by carlosuc3m 17
  • Training Obj. Detection with custom datasets

    Training Obj. Detection with custom datasets

    Question

    Hi, my question is... While I have successfully executed the training for obj. detection following the TrainPikachu class but I am getting this result from the console: Screen Shot 2020-10-05 at 11 04 16

    As you will notice, the classAccuracy and boundingBoxError are empty or has no value at all.

    Then I train the model for 8 epochs and tried to test it but am getting no detection.

    I have 3 classes to identify and my Shape is: Shape inputShape = new Shape(arguments.getBatchSize(), 3, 800, 1144);

    My index.file is like this:

    {"IMG_5237.jpg":[["0","0.085625","0.07473776223776224","0.043750000000000004","0.030594405594405596"],
    ["0","0.35125","0.28496503496503495","0.055","0.04020979020979021"],
    ["1","0.26875","0.3618881118881119","0.0575","0.033216783216783216"],
    ["0","0.7975","0.4602272727272727","0.0575","0.03583916083916084"],
    ["0","0.08750000000000001","0.6831293706293706","0.065","0.039335664335664336"],
    ["2","0.114375","0.5607517482517482","0.07875","0.028846153846153848"],
    ["2","0.115","0.40384615384615385","0.085","0.027972027972027972"],
    ["2","0.1125","0.3395979020979021","0.08","0.028846153846153848"],
    ["2","0.11375","0.23251748251748253","0.085","0.02972027972027972"],
    ["2","0.106875","0.04020979020979021","0.08125","0.026223776223776224"]], ...}
    

    So meaning I have multiple bounding boxes annotated in a single image. the first index is the class name then the second to last are the bounding boxes.

    On getting the Record, I am using the same method as with TrainPikachu:

    @Override
    protected Record get(NDManager manager, long index) throws IOException {
        int idx = Math.toIntExact(index);
        NDList d = new NDList(ImageFactory.getInstance()
                .fromFile(imagePaths.get(idx))
                .toNDArray(manager, flag));
        NDArray label = manager.create(labels.get(idx));
        NDList l = new NDList(label.reshape(new Shape(1).addAll(label.getShape())));
        return new Record(d, l);
    }
    

    Then this is how I prepare the dataset that is close to the PikachuDetection class:

    try (Reader reader = Files.newBufferedReader(indexFile)) {
        Type mapType = new TypeToken<Map<String, List<String[]>>>() {}.getType();
        Map<String, List<String[]>> metadata = JsonUtils.GSON.fromJson(reader, mapType);
        for (Map.Entry<String, List<String[]>> entry : metadata.entrySet()) {
            String imgName = entry.getKey();
            for (String[] item : entry.getValue()) {
                float[] labelArray = new float[5];
                // Class label
                labelArray[0] = Float.parseFloat(item[0]);
    
                // Bounding box labels
                labelArray[1] = Float.parseFloat(item[1]);
                labelArray[2] = Float.parseFloat(item[2]);
                labelArray[3] = Float.parseFloat(item[3]);
                labelArray[4] = Float.parseFloat(item[4]);
                labels.add(labelArray);
            }
            imagePaths.add(usagePath.resolve(imgName));
        }
    }
    

    Please let me know if I am doing something that is very different from the TrainPikachu example of why my trained model doesn't detect any object. Thank you in advance.

    question 
    opened by androuino 17
  • Modularize Maven ai.djl dependency by adding module-info.java

    Modularize Maven ai.djl dependency by adding module-info.java

    Description

    I'm trying to package my code into an executable with Maven via jpackage. However, I get an error during the jlink step of the process: Error: automatic module cannot be used with jlink: ai.djl.api from file ai.djl.api is an automatic module, meaning it cannot be linked. I know I can work around this by manually generating the module-info file, but doing so isn't easy.

    Will this change the current api? How? No

    Who will benefit from this enhancement? Anyone looking to package a Java application that uses the DJL API.

    References

    Stackover flow instructions on generating module-info.java from automatic module

    enhancement 
    opened by davpapp 0
  • [integration] Increase test coverage for NDArrays.java

    [integration] Increase test coverage for NDArrays.java

    1. Fixes contentEquals bug for boolean type
    2. Add NDUtilsTest

    Description

    Brief description of what this PR is about

    • If this change is a backward incompatible change, why must this change be made?
    • Interesting edge cases to note here
    opened by frankfliu 1
  • optionally close orphaned NDArrays using Java garbage collection

    optionally close orphaned NDArrays using Java garbage collection

    Description

    This PR is a solution suggestion to https://github.com/deepjavalibrary/djl/issues/2210.

    A striking example of the impact shows the following figure (to reproduce see https://github.com/deepjavalibrary/djl/issues/2210):

    grafik

    Design considerations: DJLDiscussionInputVersion4.pdf

    Opt-in: SwitchGarbageCollection.on();

    The implementation here is done for PyTorch, but the solution approach is general.

    opened by enpasos 6
  • Failed to download libraries

    Failed to download libraries

    Description

    Some files are failing to download after updating pytorch-engine to 0.20.0, the files aren't on your cloud instances so DJL just throws an error

    Expected Behavior

    Downloads properly

    Error Message

    ai.djl.engine.EngineException: Cannot download jni files: https://publish.djl.ai/pytorch/1.9.1/jnilib/0.20.0/linux-x86_64/cu111/libdjl_torch.so
    	at ai.djl.pytorch.jni.LibUtils.downloadJniLib(LibUtils.java:515)
    	at ai.djl.pytorch.jni.LibUtils.findJniLibrary(LibUtils.java:252)
    	at ai.djl.pytorch.jni.LibUtils.loadLibrary(LibUtils.java:80)
    	at ai.djl.pytorch.engine.PtEngine.newInstance(PtEngine.java:54)
    	at ai.djl.pytorch.engine.PtEngineProvider.getEngine(PtEngineProvider.java:40)
    	at ai.djl.engine.Engine.getEngine(Engine.java:186)
    	at ai.djl.engine.Engine.getInstance(Engine.java:141)
    Caused by: java.io.FileNotFoundException: https://publish.djl.ai/pytorch/1.9.1/jnilib/0.20.0/linux-x86_64/cu111/libdjl_torch.so
    	at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1993)
    	at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1589)
    	at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:224)
    	at java.base/java.net.URL.openStream(URL.java:1161)
    	at ai.djl.util.Utils.openUrl(Utils.java:459)
    	at ai.djl.util.Utils.openUrl(Utils.java:443)
    	at ai.djl.pytorch.jni.LibUtils.downloadJniLib(LibUtils.java:509)
    	... 12 more
    

    How to Reproduce?

    I've changed a simple application from

        implementation("ai.djl.pytorch:pytorch-engine:0.16.0")
        implementation("ai.djl.pytorch:pytorch-native-auto:1.9.1")
    

    to

        implementation("ai.djl.pytorch:pytorch-engine:0.20.0")
        implementation("ai.djl.pytorch:pytorch-native-auto:1.9.1")
    

    Steps to reproduce

    Just launch a simple program with this line to initiate the process to load the native libraries

            Engine.getInstance()
    

    What have you tried to solve it?

    These are missing files on your servers I assume, so nothing can be really done other than rollback...

    Environment Info

    N/A

    bug 
    opened by waicool20 5
  • Add spark extension docker image

    Add spark extension docker image

    Description

    Brief description of what this PR is about

    Add spark extension docker image:

    1. Use the EMR on EKS image as base image
    2. Add DJL library jars in the spark path

    Refer: https://quip-amazon.com/JJuXAm0mEiXl/Releasing-DJL-Spark-Container#temp:C:DdB07eaedeb6b7b4e3d8d35f054d

    opened by xyang16 0
  • ai.djl.translate.TranslateException: ai.djl.engine.EngineException: isTuple()INTERNAL ASSERT FAILED at

    ai.djl.translate.TranslateException: ai.djl.engine.EngineException: isTuple()INTERNAL ASSERT FAILED at "C:\\actions-runner\\_work\\pytorch\\pytorch\\builder\\windows\\pytorch\\aten\\src\\ATen/core/ivalue_inl.h":1916, please report a bug to PyTorch. Expected Tuple but got String

    ai.djl.translate.TranslateException: ai.djl.engine.EngineException: isTuple()INTERNAL ASSERT FAILED at "C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen/core/ivalue_inl.h":1916, please report a bug to PyTorch. Expected Tuple but got String

    at ai.djl.inference.Predictor.batchPredict(Predictor.java:189)
    at ai.djl.inference.Predictor.predict(Predictor.java:126)
    at com.cv.jarvis.predict.PredictApplicationTests.contextLoads(PredictApplicationTests.java:53)
    at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
    at java.base/java.lang.reflect.Method.invoke(Method.java:578)
    at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:727)
    at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
    at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
    at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156)
    at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147)
    at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:86)
    at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
    at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
    at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
    at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
    at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
    at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
    at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92)
    at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86)
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:217)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:213)
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:138)
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:68)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
    at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
    at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
    at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
    at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
    at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
    at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35)
    at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
    at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54)
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:147)
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:127)
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:90)
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:55)
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:102)
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:54)
    at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114)
    at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86)
    at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86)
    at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:53)
    at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:57)
    at com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38)
    at com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11)
    at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35)
    at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:235)
    at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:54)
    

    Caused by: ai.djl.engine.EngineException: isTuple()INTERNAL ASSERT FAILED at "C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen/core/ivalue_inl.h":1916, please report a bug to PyTorch. Expected Tuple but got String at ai.djl.pytorch.jni.PyTorchLibrary.moduleForward(Native Method) at ai.djl.pytorch.jni.IValueUtils.forward(IValueUtils.java:47) at ai.djl.pytorch.engine.PtSymbolBlock.forwardInternal(PtSymbolBlock.java:133) at ai.djl.nn.AbstractBaseBlock.forward(AbstractBaseBlock.java:77) at ai.djl.nn.Block.forward(Block.java:123) at ai.djl.inference.Predictor.predictInternal(Predictor.java:140) at ai.djl.inference.Predictor.batchPredict(Predictor.java:180) ... 70 more

    According to the official website demo, why do you report an error

    question more-information-needed 
    opened by yy-jarvis 1
Releases(v0.20.0)
  • v0.20.0(Dec 1, 2022)

    Key Features

    • Upgrades and enhancements for Engines
      • Upgrades PyTorch to 1.13.0 (#2157)
      • Add support for Apple's Metal Performance Shaders (MPS) in PyTorch (#2037)
      • Add system property to config GraphExecutorOptimize (#2156)
      • Upgrades ONNXRuntime to 1.13.1 (#2115)
      • Upgrades Paddle to 2.3.2 (#2116)
      • Upgrades TensorFlow to 2.7.4 (#2121)
      • Upgrades HuggingFace tokenizer version to 0.13.1 (#2127)
      • Upgrades XGBoost to 1.7.1 (#2143)
    • DJLServing
      • Adds large model inference support with MPI mode (#291)
      • Adds built-in DeepSpeed handler (#292)
      • Publishes PaddlePaddle docker image (#342)
    • Adds TabNet Training (#2057)
    • Publishes DJL Zero (#2091)
    • Adds Spark extension (#2162)
    • Introduces several improvements for timeseries extension
    • Adds ImageFeatureExtractor example and resnet base model to model zoo

    Enhancement

    • Introduces several enhancements for timeseries extension:
      • Adds probability distribution support for timeseries (#2025)
      • Add time series dataset support for timeseries package (#2026)
      • Update M5Forecast dataset and its unittest (#2105)
      • Add some basic block and deepAR model (#2027)
      • Enable pytorch deepar model inference in time series package (#2149)
    • Introduces several enhancement for HuggingFace tokenizer:
      • Enhance huggingface text embedding translator to support max length padding (#2049)
      • Add cli options to only validate jit model on CPU (#2052)
      • Add batch decoding methods for tokenizers (#2154)
    • Adds new models to DJL model zoo:
      • Adds TabNet model for tabular dataset in modelzoo (#2036)
      • Adds yolo5s to OnnxRuntime model zoo (#2046)
      • Object Detection (#1930)
      • Adds image classification resnet18 base model to model zoo (#2079)
    • DJL API improvements:
      • Adds Sparsemax block (#2028)
      • Updates the SemanticSegmentationTranslator (#2032)
      • Creates Ensembleable (#2043)
      • Handle error when forget to initialize a child block (#2045)
      • Adds draw mask for BitMapWrapper (#2071)
      • Allows show NDArray content in debugger (#2078)
      • Rename transparency to opacity in CategoryMask (#2081)
      • Allows to show NDArray content in Debugger 2 (#2080)
      • Transfer learning with pytorch engine on fresh fruit dataset (#2070)
      • Ensure GradientCollector can clear gradients (#2101)
      • Handles conflict JNA package issue (#2118)
      • Adds Multiplication block (#2110)
      • Allows non-ServingTranslatorFactory for DJLServing (#2148)
      • Adds cumprod operator (#2152)
      • Adds Randperm on PyTorch and MxNet (#2084)
      • Creates translator options (#2145)
    • CI improvements:
      • Add Mac M1 build (#2039)
      • Publishes serving tar and zip (#2014)
      • Uploads djl-bench release artifacts to S3 (#2020)
      • Upgrade deprecated github actions (#2119)
      • Upgrade github actions to latest version (#2122)
      • Compile JNI only when file changes (#2161)
      • Speed up continuous build by not uploading jacoco report (#2166)
      • Respect -SNAPSHOT version in jar manifest (#2177)
      • Allows JNI to be compiled on headless jdk (#2098)
      • Move model zoo download test to canary (#2169)
      • Upgrades PyTorch for Android to 1.13.0 (#2171)
      • Add some unit tests (#2063)
      • Test accumulating gradient collector (#2111)
      • Refactor unit test TestRequirements, add missing TestRequirements (#2120)
    • Upgrade protobuf version to 3.20.2 (#2035)
    • Update deeplabv3 model zoo metadata (#2051)
    • Remove String tensor limitation for model output (#2056)
    • Disables mapLocation when using MPS device (#2061)
    • Adds disablePerSessionThreads option to model loading for ONNXRuntime (#2104)
    • LightGBM inference result matches input type (#2129)
    • Apply no_optimizer_guard only for Android (#2153)
    • Update dependency version (#2176)
    • Reduce nested exception level (#2181)

    Documentation and Examples

    • Updates Semantic Segmentation app (#263)
    • Adds the object detection app demo, use onnxruntime engine (#266)
    • Adds stable diffusion demo (#269)
    • Adds Spark extension example (#272)
    • Adds DJLServing Postman examples (#276)
    • Adds DJLServing Java client demo (#277)
    • Adds DJLServing Python client demo (#278)
    • Update README.md (#2010)
    • Update dependency docs for timeseries package (#2004)
    • Update javadoc links (#2017)
    • Use latest javadoc links (#2021)
    • Info added (#2022)
    • Add Mac M1 info in docs (#2040)
    • Some doc fixes (#2042)
    • Improve memory management and batchify docs (#2076)
    • Move serving docs to top level and reorganize (#2100)
    • Update docs top level memu (#2102)
    • Change timeseries dataset source example and add test (#2109)
    • Fix a document issue (#2114)
    • Update dependency document (#2134)
    • Upgrade pytorch 1.13.0 documents (#2158)
    • Add readme for TransferFreshFruit (#2160)
    • Made the sites copyright year dyanamic (#2188)

    Breaking change

    • NDArray.toDebugString() signature has been changed (#2078)

    Bug Fixes

    • Fixes flooded warning message in PyTorch (#2136)
    • Fixes youtube link in quick start (#2012)
    • Fixes dlr-native build script (#2015)
    • Adds missing condition for benchmark release (#2023)
    • Fixes folder not exist bug when use external pytorch native library (#2033)
    • Fixes bug in OD TF saved model example (#2050)
    • Fixes MXNet engine cu112 document (#2066)
    • Fixes CategoryMask (#2073)
    • Fixes ames and various tabular improvements (#2054)
    • Fixes randomColor (#2075)
    • Fixes remove warning spam during training (#2097)
    • Adds missing test requirements to opencv test (#2108)
    • Fixes android crash issue (#2113)
    • Avoid static setup/tearDown in testng (#2126)
    • Add missing dependency to timeseries (#2128)
    • Fixes mac os build failure (#2131)
    • Fixes char offset (#2137)
    • Adds repo_url in order to fix broken edit links (#2140) (#2141)
    • Fixes format bug and change the name of retrain (#2147)
    • Fixes ndarry operator warnings show up in Jupyter notebook (#2150)
    • Fixes HfModelZoo NPE bug with shadowjar (#2163)
    • Fixes Temporary File Information Disclosure Vulnerability (#2164)
    • Fixes movielens download issue (#2167)
    • Fixes Yolov3 javadoc warning (#2168)
    • CVE-2007-4559 Patch (#2189)

    Contributors

    • @asbachb
    • @Carkham
    • @demq
    • @dependabot
    • @frankfliu
    • @JLLeitschuh
    • @KexinFeng
    • @lanking520
    • @patins1
    • @siddvenk
    • @tosterberg
    • @warthecatalyst
    • @wxm2018
    • @xyang16
    • @ylwu-amzn
    • @zachgk

    New Contributors

    • @dependabot made their first contribution in (#2035)
    • @ylwu-amzn made their first contribution in (#2049)
    • @tosterberg made their first contribution in (#2097)
    • @asbachb made their first contribution in (#2128)
    • @JLLeitschuh made their first contribution in (#2164)

    Full Changelog: https://github.com/deepjavalibrary/djl/compare/v0.19.0...v0.20.0

    Source code(tar.gz)
    Source code(zip)
  • v0.19.0(Sep 14, 2022)

    Key Features

    • Creates new LightGBM engine (#1895)
    • Upgrades and enhancements for Engines
      • Upgrades PyTorch to 1.12.1 (#1894)
      • Upgrades ONNXRuntime to 1.12.1 (#1879)
      • Upgrades Apache MXNet to 1.9.1 (#1898)
      • Publishes new xgboost-gpu package to maven (#1918)
      • Adds ARM support for ONNXRuntime (#1856)
      • Disable autograd by default when PyTorch engine start (#1872)
    • Introduces several enhancement for HuggingFace tokenizer
      • Introduces HuggingFace model zoo (#1984)
      • Adds a few built-in Translators for HuggingFace NLP models
      • Adds macOS M1 support for HuggingFace tokenizer
      • Adds ARM support for HuggingFace tokenizer
      • Adds centos 7 support for HuggingFace tokenizer (#1874)
      • Adds decode API for HuggingFace tokenizer (#1843)
      • Adds padding and truncation support for HuggingFace tokenizer (#1870)
      • Support stride in tokenizers (#2006)
    • Introduces time series extension (#1903)
    • Adds new Audio API and improves audio extension (#1974)
    • Adds Android support for ONNXRuntime (#1844)
    • JDK18 support (#1892)
    • Adds python script to import HuggingFace model into DJL model zoo (#1835)
    • DJLServing
      • Adds management console plugin, which allows user manage models with web UI (#205)
      • Adds KServe plugin (#177)
      • Publishes DeepSpeed docker image to dockerhub (#223)

    Enhancement

    • Adds a few more built-in Translators:
      • Adds HuggingFace QuestionAnsweringTranslator (#1828)
      • Adds HuggingFace FillMaskTranslator (#1876)
      • Adds HuggingFace TokenClassificationTranslator (#1906)
      • Adds HuggingFace TextClassificationTranslator (#1983)
      • Adds HuggingFace TextEmbeddingTranslator (#1953)
      • Adds speech recognition translator (#1899)
    • Adds new models to DJL model zoo:
      • Adds PyTorch deeplabvs model into DJL model zoo (#1818)
      • Adds MobileNetV1 into model zoo (#1817)
    • Image handling enhancement:
      • Improves ImageFactory to allow convert float32 NDArray to Image. (#1814)
      • Handle both HWC and CHW image (#1833)
    • DJL API improvements:
      • Adds NDArray normalize() operator (#1924)
      • Adds DeferredTranslatorFactory to let serving.properties take effect (#1868)
      • Makes PtBertQATranslator compatible with huggingface model (#1827)
      • Improves debug log for model loading options. (#1825)
      • Allows to load block only model for PyTorch (#1831)
      • Adds IdentityBlockFactory for demo/test purpose (#1854)
      • Support queryString for JarRepository (#1842)
      • Set arguments in serving.properties as model properties (#1853)
      • Allow overriding special token flags in encode and decode methods (#1855)
      • Adds support for intermediate sequential block results (#1943)
      • Adds load SentencePiece model from InputStream (#1949)
      • Allows use cached PyTorch native libraries in offline mode by caching "files.txt". (#1982)
      • Makes Encoding class constructor protected. (#1945)
      • Adds string tensor support for PyTorch (#1968)
      • Adds Loss function: Coverage.java (#1653)
      • Adds Loss function: QuantileLoss.java (#1652)
      • Validate data type for NDArray.set(Buffer) API (#1975)
      • Adds offline mode to to ensure not download engine files from network (#1987)
      • Adds encodeDual support for HuggingFace tokenizer (#1826)
      • Bulk batch creation and array indexing on mxnet engine (#1869)
      • Adds NDArray gammaln and sample distribution function support. (#1990)
      • Padding when the size of input is 2 in LSTM (#2000)
      • Creates a SystemNDManager interface (#1888)
    • Adds python script to import huggingface model into DJL model zoo
      • Added fill-mask support for converting huggingface model to model zoo (#1849)
      • Adds support for converting huggingface token-classification models (#1902)
      • Adds support for converting huggingface sentence-similarity models (#1913)
      • Adds support for converting huggingface text-classification models (#1972)

    Documentation and Examples

    • Adds Neural machine translation example (#1851)
    • Adds New Bert example using Goemotions (#1682)
    • Adds Semantic Segmentation example (#1808)
    • Adds tokenizer readme for usage (#1981)
    • Updates troubleshooting.md to remove -native-auto package (#1793)
    • Document PYTORCH_PRECXX11 usage in README (#1807)
    • Immutable array output from InferenceMode PyTorch (#1822)
    • Fixes NDIndex javadoc issue (#1841)
    • Updates pose estimation example to detect joints for all people (#2002)
    • Adds Semantic segmentation and Speech recognition to README (#2003)
    • Updates links in README (#2005)
    • Adds an example of time series model inference (#1971)

    Breaking change

    • NDManager.vaildateBufferSize() has been renamed to NDManager.validateBuffer()
    • Remove unnecessary DeviceType interface (#1978)

    Bug Fixes

    • Adds missing text_embedding application in Application.of() (#1917)
    • Fixes capped manager bug for TensorFlow (#1952)
    • Fixes NDArray.set() bug (#1789)
    • Fixes breaking behavior for NDIndex in 0.18.0 (#1801)
    • Backward compatible with Apache MXNet indexing. (#1802)
    • Fixes OrtNDArray double close issue (#1809)
    • Fixes ImageFactory.fromNDArray() bug (#1824)
    • Fixes NDArrayAdapter toDevice() and toType() behavior (#1839)
    • Fixes the parsing issue (#1857)
    • Fixes OrtNDArray double free issue (#1861)
    • Fixes memory leak when using NDManager.newBaseManager() (#1887)
    • Fixes PyTorch download library fallback to CPU bug (#1951)
    • Fixes bug in Criteria (#1964)
    • Fixes issue in TrainMnistWithLSTM (#1965)
    • Fixes closing error in Apache MXNet when indexing results in an empty array (#1966)
    • Fixes path parsing bug on Windows (#1985)
    • Fixes memory leak in Apache MXNet layerNorm() (#1993)
    • Fix DynamicBuffer position error when expand for the first time (#2007)
    • Fix some bugs in pytorch based examples (#2009)

    Contributors

    • @925781609
    • @bryanktliu
    • @Carkham
    • @demq
    • @frankfliu
    • @gforman44
    • @JohnDoll2023
    • @KexinFeng
    • @Konata-CG
    • @lanking520
    • @oyy2000
    • @patins1
    • @pdradx
    • @siddvenk
    • @takanori-ugai
    • @warthecatalyst
    • @wxm2018
    • @xyang16
    • @zachgk

    New Contributors

    • @bryanktliu made their first contribution in (#1828)
    • @xyang16 made their first contribution in (#1852)
    • @wxm2018 made their first contribution in (#1844)
    • @925781609 made their first contribution in (#1887)
    • @demq made their first contribution in (#1945)
    • @Carkham made their first contribution in (#1903)
    • @gforman44 made their first contribution in (#1653)
    • @takanori-ugai made their first contribution in (#2000)

    Full Changelog: https://github.com/deepjavalibrary/djl/compare/v0.18.0...v0.19.0

    Source code(tar.gz)
    Source code(zip)
  • v0.18.0(Jul 11, 2022)

    Key Features

    • Adds macOS M1 chip support for PyTorch https://github.com/deepjavalibrary/djl/pull/1656, https://github.com/deepjavalibrary/djl/pull/1696
    • JDK 17 support https://github.com/deepjavalibrary/djl/pull/1672
    • Full support of PyTorch Get Indexing for NDArrays https://github.com/deepjavalibrary/djl/pull/1719
    • Full support of PyTorch Set Indexing for NDArrays https://github.com/deepjavalibrary/djl/pull/1755
    • Updates Dataset documentation https://github.com/deepjavalibrary/djl/pull/1686
    • Moves djl-bench to DJL Serving https://github.com/deepjavalibrary/djl/pull/1743
    • Engines and Extensions
      • TensorFlow 2.7.0 https://github.com/deepjavalibrary/djl/pull/1674
      • New djl-audio extension https://github.com/deepjavalibrary/djl/pull/1681
      • Adds GPU support for XGBoost https://github.com/deepjavalibrary/djl/pull/1680
      • tokenizers 0.12.0 https://github.com/deepjavalibrary/djl/pull/1739
      • sentencepiece 0.1.96 https://github.com/deepjavalibrary/djl/pull/1745
      • TensorRT 8.4.1 https://github.com/deepjavalibrary/djl/pull/1758
    • Newly Added Datasets
      • Goemotions dataset https://github.com/deepjavalibrary/djl/pull/1598
      • Daily Delhi Climate Dataset https://github.com/deepjavalibrary/djl/pull/1667
      • Tablesaw Dataset https://github.com/deepjavalibrary/djl/pull/1679
      • Universal Dependencies Corpus for English https://github.com/deepjavalibrary/djl/pull/1595
      • Movielens 100k dataset https://github.com/deepjavalibrary/djl/pull/1718

    Enhancement

    • Increases build version to 0.18.0 https://github.com/deepjavalibrary/djl/pull/1645
    • Support of take from pytorch https://github.com/deepjavalibrary/djl/pull/1627
    • Upgrades JNA to 5.11.0 https://github.com/deepjavalibrary/djl/pull/1655
    • Improves ServingTranslator output handling https://github.com/deepjavalibrary/djl/pull/1654
    • Adds width/height conversion to ObjectDetection https://github.com/deepjavalibrary/djl/pull/1651
    • Add openCV find rectangle method to improve PaddleORC performance https://github.com/deepjavalibrary/djl/pull/1662
    • Removes unnecessary logics in Paddle https://github.com/deepjavalibrary/djl/pull/1676
    • Adds Cyclical Tracker https://github.com/deepjavalibrary/djl/pull/1671
    • Adds support of take on MXNet engine https://github.com/deepjavalibrary/djl/pull/1649
    • Implements GhostBatchNorm https://github.com/deepjavalibrary/djl/pull/1666
    • Allows indexer to attach specific manager https://github.com/deepjavalibrary/djl/pull/1688
    • Upgrades android module to use DJL 0.18.0 https://github.com/deepjavalibrary/djl/pull/1693
    • Uses pytorch to test API and aws-ai module https://github.com/deepjavalibrary/djl/pull/1695
    • Avoid download cudf dependency for XGBoost at build time https://github.com/deepjavalibrary/djl/pull/1694
    • Bumps up versions https://github.com/deepjavalibrary/djl/pull/1691
    • Refactors ServingTranslatorFactory https://github.com/deepjavalibrary/djl/pull/1702
    • Adds "capped" state to NDManager https://github.com/deepjavalibrary/djl/pull/1683
    • Upgrades NDK version to 21.1.6352462 https://github.com/deepjavalibrary/djl/pull/1707
    • Adds LinearCollection block https://github.com/deepjavalibrary/djl/pull/1658
    • Adds android test code https://github.com/deepjavalibrary/djl/pull/1714
    • Changes DJL repo names from aws-samples https://github.com/deepjavalibrary/djl/pull/1716
    • Adds serving deb file publish for CI https://github.com/deepjavalibrary/djl/pull/1721
    • Upgrades codeql github action to v2 https://github.com/deepjavalibrary/djl/pull/1730
    • Fixes publish serving deb https://github.com/deepjavalibrary/djl/pull/1725
    • Adds ai.djl.audio and ai.djl.tablesaw to BOM https://github.com/deepjavalibrary/djl/pull/1728
    • Upgrades java formatter to 1.15.0 https://github.com/deepjavalibrary/djl/pull/1727
    • Adds name to LambdaBlock https://github.com/deepjavalibrary/djl/pull/1726
    • Adds disable static option in MXNet to allow some model running https://github.com/deepjavalibrary/djl/pull/1735
    • Improves Criteria.toBuilder() api https://github.com/deepjavalibrary/djl/pull/1741
    • Fixes serving publish github actions https://github.com/deepjavalibrary/djl/pull/1742
    • Enables better textual description of neural net https://github.com/deepjavalibrary/djl/pull/1720
    • Ignores hidden files for nested model directory https://github.com/deepjavalibrary/djl/pull/1754
    • Creates action to auto-close issues without response https://github.com/deepjavalibrary/djl/pull/1751
    • Builds jni for aarch64 https://github.com/deepjavalibrary/djl/pull/1756
    • Removes unnecessary packages from tensorrt dockerfile https://github.com/deepjavalibrary/djl/pull/1760
    • Adds log for custom Translator loading https://github.com/deepjavalibrary/djl/pull/1761
    • Stores indices with batch https://github.com/deepjavalibrary/djl/pull/1750
    • Adds put feature with linear indexing on PyTorch engine https://github.com/deepjavalibrary/djl/pull/1749
    • Adds NDList to IValue unit test https://github.com/deepjavalibrary/djl/pull/1762
    • Makes tensorflow NDArray always dense https://github.com/deepjavalibrary/djl/pull/1763
    • JDK version updated https://github.com/deepjavalibrary/djl/pull/1767
    • Adds IValue Dict(str, IValue) support https://github.com/deepjavalibrary/djl/pull/1765
    • Creates tabular dataset https://github.com/deepjavalibrary/djl/pull/1699
    • Creates PreparedFeaturizer https://github.com/deepjavalibrary/djl/pull/1700
    • Normalizes Numeric Featurizer https://github.com/deepjavalibrary/djl/pull/1701
    • Adds support for registerCustomOpLibrary for ONNXRuntime. https://github.com/deepjavalibrary/djl/pull/1771
    • Implements inverse operation https://github.com/deepjavalibrary/djl/pull/1768
    • Supports Image output for ImageServingTranslator https://github.com/deepjavalibrary/djl/pull/1772
    • Allows user specify model name in serving.properties file https://github.com/deepjavalibrary/djl/pull/1780
    • Adds model zoo implementation https://github.com/deepjavalibrary/djl/pull/1781
    • Change the sagemaker model to s3 https://github.com/deepjavalibrary/djl/pull/1769
    • Improvements to image coloring https://github.com/deepjavalibrary/djl/pull/1784
    • Updates bert classification notebook to reflect changes in CSVDataset https://github.com/deepjavalibrary/djl/pull/1786
    • Paddle model zoo should not have compile time dependency on opencv https://github.com/deepjavalibrary/djl/pull/1785

    Documentation and Examples

    • Updates README for 0.17.0 Release https://github.com/deepjavalibrary/djl/pull/1646
    • Increases DJL Version for main branchhttps://github.com/deepjavalibrary/djl/pull/1644
    • Fixes broken and redirected links https://github.com/deepjavalibrary/djl/pull/1647
    • Clarifies typo in example documentation https://github.com/deepjavalibrary/djl/pull/1685
    • Fixes javadoc error in JDK 1.8 https://github.com/deepjavalibrary/djl/pull/1698
    • Update description for latest javadoc location https://github.com/deepjavalibrary/djl/pull/1708
    • Creates README for DJL Android PyTorch 1.11 builds https://github.com/deepjavalibrary/djl/pull/1704
    • Adds serving to docs site https://github.com/deepjavalibrary/djl/pull/1715
    • Fixes broken javadoc links in jupyter notebooks https://github.com/deepjavalibrary/djl/pull/1722
    • Readme updates for PyTorch 1.11 https://github.com/deepjavalibrary/djl/pull/1709
    • Updates CVSDataset example README file https://github.com/deepjavalibrary/djl/pull/1729
    • Updates document to use MXNet 1.9.0 https://github.com/deepjavalibrary/djl/pull/1737
    • Adds documentation on loading TF extension libraries for running certa… https://github.com/deepjavalibrary/djl/pull/1776
    • Adds semantic segmentation example https://github.com/deepjavalibrary/djl/pull/1764

    Breaking Changes

    The following changes to api.djl.basicdataset.tabular may cause backwards incompatibility:

    • Features and Featurizers have been refactored out of the CSVDataset class. The are now present in ai.djl.basicdataset.tabular.utils
    • CSVDataset now extends a new abstract class, TabularDataset
    • api.djl.basicdataset.utils.DynamicBuffer implementation has moved to api.djl.basicdataset.tabular.utils.DynamicBuffer

    Bug Fixes

    • [TensorFlow] fix GPU memory leak https://github.com/deepjavalibrary/djl/pull/1648
    • [tensorrt] Fixes native library path https://github.com/deepjavalibrary/djl/pull/1650
    • Fixes bug in NDArray.oneHot() API https://github.com/deepjavalibrary/djl/pull/1661
    • Fix errors in "getIoU" function https://github.com/deepjavalibrary/djl/pull/1687
    • Follow symlinks when loading models. https://github.com/deepjavalibrary/djl/pull/1692
    • [pytorch] Fixes model loading bug for 1.11.0 https://github.com/deepjavalibrary/djl/pull/1705
    • Ensure PreparedOneHotStringFeaturizer encodes categorical mappings co… https://github.com/deepjavalibrary/djl/pull/1723
    • [tensorflow] Avoid NPE in TfEngine https://github.com/deepjavalibrary/djl/pull/1734
    • [m1] Fix test failure on macOS M1 machine by @frankfliu in https://github.com/deepjavalibrary/djl/pull/1777

    Contributors

    @dandansamax @DiaaAj @frankfliu @WHALEEYE @patins1 @JohnDoll2023 @KexinFeng @Konata-CG @pdradx @lanking520 @siddvenk @LanAtGitHub @warthecatalyst @zachgk @freemanliu @liumingxiy

    New Contributors

    • @Konata-CG made their first contribution in https://github.com/deepjavalibrary/djl/pull/1598
    • @pdradx made their first contribution in https://github.com/deepjavalibrary/djl/pull/1661
    • @JohnDoll2023 made their first contribution in https://github.com/deepjavalibrary/djl/pull/1685
    • @liumingxiy made their first contribution in https://github.com/deepjavalibrary/djl/pull/1687
    • @LanAtGitHub made their first contribution in https://github.com/deepjavalibrary/djl/pull/1679
    • @freemanliu made their first contribution in https://github.com/deepjavalibrary/djl/pull/1692
    • @DiaaAj made their first contribution in https://github.com/deepjavalibrary/djl/pull/1666
    • @warthecatalyst made their first contribution in https://github.com/deepjavalibrary/djl/pull/1767
    • @siddvenk made their first contribution in https://github.com/deepjavalibrary/djl/pull/1718

    Full Changelog: https://github.com/deepjavalibrary/djl/compare/v0.17.0...v0.18.0

    Source code(tar.gz)
    Source code(zip)
  • v0.17.0(May 11, 2022)

    Key Features

    • Adds linux AArch64 support for PyTorch
    • Upgrades Engine Releases
      • XGBoost version 1.6.0 (#1624)
      • PaddlePaddle version 2.2.2 (#1601)
      • ONNXRuntime version 1.11.0 (#1602)
      • PyTorch version 1.11.0 (#1583)
      • Apache MXNet version 1.9.0 (#1429)
    • Newly Added Datasets
      • PennTreebank dataset (#1580)
      • WikiText-2 dataset (#1545)
    • Support freezing parameters for transfer learning (#1544)

    Enhancement

    • Upgrade djl-bench to 0.15.0 (#1476)
    • Add SessionOptions support for OnnxRuntime (#1479)
    • Device Name parsing (#1490)
    • arena allocator setter (#1510)
    • Upgrade native build mxnet version (#1517)
    • Refactor ml extension support (#1521)
    • Parse default device with Device.fromName (#1529)
    • Upgrade dependency versions (#1533)
    • release Update document version to 0.16.0 (#1536)
    • Bump up version to 0.17.0 (#1537)
    • Building DJL For aarch64 (#1526)
    • api Make resource loading compatible with java9 module (#1541)
    • pytorch Allows multiple native jars to package into fat jar (#1543)
    • Add document with installation instructions (#1548)
    • Separate AbstractSymbolBlock from AbstractBlock (#1555)
    • Add better error message for libstdc++ tf errors (#1570)
    • basicdataset Add Stanford Question Answering Dataset (#1554)
    • xgb Set default missing value to NaN (#1571)
    • allow empty batch data (#1569)
    • pytorch Allows load libtroch from pip installation package (#1577)
    • HF Tokenizer: get charspans (#1584)
    • benchmark Allows benchmark run on aarch64 for PyTorch (#1591)
    • add pytorch cuDNN acceleration (#1592)
    • add troubleshoot issue (#1600)
    • add paddle CU110 (#1604)
    • Update badge (#1610)
    • ONNXRuntime add tensorRT option (#1611)
    • api Refactor metrics API (#1613)
    • api Fixes metric dimension (#1614)
    • bom Update bom dependency (#1615)
    • pytorch Use precxx11 build for aarch64 native library (#1619)
    • pytorch use precxx11 for aarch64 (#1620)
    • Update inference_performance_optimization.md (#1621)
    • Fix testFreezeParameters for multi-gpu (#1623)
    • Support gather of pytorch (#1622)

    Documentation and Examples

    • Update README for 0.15.0 release (#1477)
    • Update README to use 0.16.0-SNAPSHOT version (#1486)
    • typo fix (#1519)
    • docs Fixes dataset document. (#1523)
    • jupyter remove unecessary maven import (#1540)
    • examples Update maven pom file dependency version (#1546)
    • Add release note to README (#1565)
    • update pytorch build instruction on android (#1630)
    • bump up versioning (#1633)
    • update android package versions (#1635)
    • docs Update pytorch and paddle version (#1634)

    Breaking Changes

    • Custom symbol blocks should extend AbstractSymbolBlock instead of AbstractBlock

    Bug Fixes

    • Fix flaky test for tensorflow 2.7.0 on GPU (#1475)
    • Fixes topK items for DetectedObjects and make it configurable to Classifications (#1478)
    • Fix load native library failure on Android (#1485)
    • Adding huggingface tokenizer extension to BOM (#1487)
    • Fixes #1149, fix NPE bug (#1492)
    • Fix djl site vue version (#1495)
    • Fixes MLP example code in README (#1497)
    • Fixes jni dependency in README document (#1513)
    • Fixes memory leak in hybrid engine (#1518)
    • Fix the version issue reported from get-pip.py (#1530)
    • Fix FastText JNI build (#1531)
    • api Fixes loading BlockFactory bug (#1547)
    • tensorflow Fixes tensorflow session always on gpu(0) bug (#1558)
    • xgb Fixes missing anonymous classes (#1572)
    • examples Fix ImageClassification invalid probability (#1575)
    • fix naming (#1581)
    • ONNXRuntime fix naming (#1608)
    • basicdataset Fixed out of bound limit (#1599)
    • api Avoid NPE in Metric.toString() (#1626)
    • integration Enable gather unit test for windows (#1638)
    • Add rpath fix to Native Publish PyTorch (#1639)
    • api Fixes JDK 18 compiler warnings (#1640)

    Contributors

    • @AKAGIwyf
    • @andreabrduque
    • @dandansamax
    • @frankfliu
    • @hd1080p
    • @KexinFeng
    • @lanking520
    • @patins1
    • @sindhuvahinis
    • @WHALEEYE
    • @zachgk

    New Contributors

    • @sindhuvahinis made their first contribution in https://github.com/deepjavalibrary/djl/pull/1519
    • @KexinFeng made their first contribution in https://github.com/deepjavalibrary/djl/pull/1530
    • @hd1080p made their first contribution in https://github.com/deepjavalibrary/djl/pull/1526
    • @dandansamax made their first contribution in https://github.com/deepjavalibrary/djl/pull/1545
    • @WHALEEYE made their first contribution in https://github.com/deepjavalibrary/djl/pull/1554
    • @patins1 made their first contribution in https://github.com/deepjavalibrary/djl/pull/1569
    • @AKAGIwyf made their first contribution in https://github.com/deepjavalibrary/djl/pull/1580

    Full Changelog: https://github.com/deepjavalibrary/djl/compare/750c153a7...v0.17.0

    Source code(tar.gz)
    Source code(zip)
  • v0.16.0(Mar 22, 2022)

    Key Features

    • Upgrades Apache MXNet engine to 1.9.0 with CUDA 11.2 support
    • Improves ONNXRuntime engine memory configurations
    • Improves fastText engine’s API
    • Fixes several critical bugs

    Enhancement

    • Upgrades Apache MXNet engine to 1.9.0 with CUDA 11.2 support (#1517)
    • Improves ONNXRuntime engine:
      • Adds arena allocator support to SessionOptions (#1510)
      • Adds SessionOptions support for OnnxRuntime (#1479)
    • Introduces several API improvements:
      • Parse default device with Device.fromName (#1529)
      • Refactor ml extension support to be consistent with DJL api (#1521)
      • Device Name parsing (#1490)
      • Make DetectedObjects topK configurable (#1478)
    • Uses JDK 11 for github actions in CI build (#1489)
    • Adds huggingface tokenizer extension to BOM (#1487)
    • Removes unnecessary github actions workflow (#1484)
    • Upgrades DJL android to 0.15.0 (#1483)

    Documentation and examples

    • Fixes outdated dataset document (#1523)
    • Fixes repository README typo (#1519)
    • Fixes PyTorch JNI dependency in README document (#1513)
    • Fixes MLP example code in README (#1497)
    • Fixes djl.ai website vue version (#1495)
    • Updates inferentia demo to DJL 0.15.0 (#210)
    • Updates android README to use 0.15.0 (#1486)
    • Publishes D2L Chinese book with latest chapters https://d2l-zh.djl.ai

    Breaking change

    • fastText specific inference and training APIs are removed, use standard DJL API instead

    Bug Fixes

    • Fixes memory leak in hybrid engine (#1518)
    • Fixes fastText JNI build (#1531)
    • Fixes the python version bug in the benchmark workflow files (#1530)
    • Fixes sentencepiece NPE bug (#1492)
    • Fixes PyTorch load native library failure on Android (#1485)
    • Fixes topK items for DetectedObjects (#1478)

    Contributors

    This release is thanks to the following contributors:

    New Contributors

    • Andréa Duque made their first contribution in https://github.com/deepjavalibrary/djl/pull/1510

    Full Changelog: https://github.com/deepjavalibrary/djl/compare/v0.15.0..v0.16.0

    Source code(tar.gz)
    Source code(zip)
  • v0.15.0(Jan 19, 2022)

    DJL v0.15.0 updates the engines PyTorch to 1.10.0, ONNXRuntime to 1.10.0, TensorFlow to 2.7.0, TensorFlowLite to 2.6.2 and introduces several new features:

    Key Features

    • Introduces Huggingface tokenizers extension which allows user to leverage high performance fast tokenizer in Java
    • Upgrades PyTorch engine to 1.10.0 with CUDA 11.3 support
    • Upgrades TensorFlow to 2.7.0 with CUDA 11.3 support
    • Upgrades ONNXRuntime engine to 1.10.0
    • Upgrades TensorFlowLite to 2.6.2
    • Provides better PyTorch engine backward compatibility support
    • Adds load model from InputStream support
    • Adds Windows support for SentencePiece
    • Removes -auto packages to simplify DJL dependencies
    • Fixes log4j CVEs

    Enhancement

    • Improves PyTorch engine:
      • Adds support to load custom build PyTorch native library from specified location
      • Adds support to use precxx11 version of PyTorch on GPU
      • Provides offline native package for older version of PyTorch to run with latest DJL (#1385)
      • Report correct engine version for custom build PyTorch native library
      • Adds Tuple support to IValue (#1436)
    • Adds support to load model from InputStream for some of the engines (#1400):
      • Adds load from InputStream for PyTorch model
      • Adds load from InputStream for TensorFlowLite model (#1402)
      • Adds load from InputStream for ONNXRuntime model (#1402)
      • Adds load from InputStream for SentencePiece (#1139)
    • Introduces several new features in djl-serving:
      • Automatically detect model’s engine
      • Reduces netty dependencies to minimize package size
      • Installs engine dependency on-demand for non-commonly used engines
      • Adds support for nested folder in model archive file
      • Released djl-serving to homebrew
    • Introduces server new features in djl-bench
      • Release djl-bench to homebrew and snapcraft
    • Improve opencv extension to expose API from opencv to end users
    • Introduces several API improvements:
      • Improves TrainingResult with toString() print out (#1369)
      • Allows to register engine at runtime (#1386)
      • Allows to dynamic add model zoo (#1397)
      • Creates IndexEvaluator and IndexLoss (#1414)
      • Introduces Huggingface Tokenizer (#1406)
    • Publishes several new docker images to dockerhub (https://hub.docker.com/u/deepjavalibrary)
      • djl-serving docker image
      • djl-serving for inferentia docker image
      • DJL windows docker image with jdk11

    Documentation and examples

    • Updates inferentia demo to support neuron runtime 2.x
    • Updates jupyter notebooks with NoBatchifyTranslator to simplify example code (#1370)
    • Updates benchmark README (#1410)
    • Fixes rank classification jupyter notebook (#1368)
    • Updates packaging model document (#1471)
    • Fixes d2l book (https://d2l.djl.ai/) image display issue (#183)
    • Translates d2l Chinese book (https://d2l-zh.djl.ai/) chapter 9 to chapter 14 to Chinese

    Breaking change

    • N/A

    Bug Fixes

    • Fixes several ci issues on Github Actions (#1358, #1362, #1363, #1364)
    • Fixes crash when IValue list input is empty (#1440)
    • Fixes NullPointException bug in getContextClassLoader() (#1445)
    • Fixes model name detection issue in loading model from jar (#1446)
    • Fixes unset numEmbeddings in TrainableWordEmbedding.java (#1450)
    • Fixes protobuf-java CVEs

    Contributors

    This release is thanks to the following contributors:

    New Contributors

    • @Lundez made their first contribution in https://github.com/deepjavalibrary/djl/pull/1412
    • @zakhio made their first contribution in https://github.com/deepjavalibrary/djl/pull/1436
    • @daofaziran1 made their first contribution in https://github.com/deepjavalibrary/djl/pull/1450

    Full Changelog

    https://github.com/deepjavalibrary/djl/compare/0855108d0...v0.15.0

    Source code(tar.gz)
    Source code(zip)
  • v0.14.0(Nov 10, 2021)

    DJL v0.14.0 updates the engines PyTorch to 1.9.1 and introduces several new features:

    Key Features

    • Upgrades PyTorch engine to 1.9.1
    • Adds support for Neuron SDK 1.16.1
    • Adds autoscale in djl-serving for AWS Inferentia model
    • Introduces OpenCV extension to provide high performance image processing
    • Adds support for older version of PyTorch engine, user now can use PyTorch 1.8.1 with latest DJL
    • Adds support for precxx11 PyTorch native library in auto detection mode
    • Adds AWS Inferentia support in djl-bench
    • Adds support for TorchServe .mar format, user can deploy TorchServe model archive in djl-serving

    Enhancement

    • Introduces several new features in djl-serving:
      • Adds autoscale feature for AWS Inferentia (#31)
      • Creates SageMaker hosting compatible docker image for AWS Inferentia (#36)
      • Adds auto detect number of neuron cores feature for AWS Inferentia (#34)
      • Adds autoscale support for SageMaker style .tar.gz model (#35)
      • Adds support to load torchserve model (#32)
      • Adds support to pip installed dependency per model (#37)
      • Adds custom environment variable support for python engine (#29)
      • Adds nested folder support in model archive file (#38)
      • Improves model status with model version support (#25)
      • Adds model warn up feature for python engine. (#23)
      • Adds WorkLoadManager.unregisterModel (#33)
      • Adds testing tool to test python model locally (#22)
      • Adds set python executable path for python engine (#21)
      • Creates Workflow for ModelServing (#26)
    • Adds OpenCV extension (#1331)
    • Introduces several new features in djl-bench:
      • Adds support for AWS Inferentia (#1329)
    • Introduces several new features in Apache MXNet engine:
      • Implements LayerNorm for Apache MXNet (#1342)
    • Introduces several new features in PyTorch engine:
      • Upgrades PyTorch to 1.9.1 (#1297)
      • Implements padding to bert tokenizer (#1328)
      • Makes pytorch-native-auto package optional (#1326)
      • Adds support to use different version of PyTorch native library (#1323)
      • Adds map_location support for load model from InputStream (#1314)
      • Makes map_location optional (#1312)
    • Introduces several new features in TensorFlow Lite engine:
      • Makes tensor-native-auto package optional (#1301)
    • Introduces several API improvements:
      • Adds support for nested folder in model archive (#1349)
      • Improves translator output error message (#1348)
      • Improves Predictor API to support predict with device (#1346)
      • Improves BufferedImageFactory.fromNDArray performance (#1339)
      • Adds support for downloading .mar file (#1338)
      • Adds debugging toString to Input and Output (#1327)
      • Refactors BERT Translator and Tokenizer (#1318)
      • Makes question answering model serving ready (#1311)
      • Refactors minMaxWorkers from ModelInfo to WorkerPool (#30)

    Documentation and examples

    Breaking change

    • PyTorch 1.9.1 no longer supports Amazon Linux 2, AL2 user has to use pytorch-native-cpu-precxx11
    • Image.Type is removed and Image.duplicate() function no longer take Image.Type as input
    • Image.getSubimage() is renamed to Image.getSubImage()
    • PaddlePaddle model loading may break due to prefix changes.

    Bug Fixes

    • Fixes 2nd inference throw exception bug (#1351)
    • Fixes calculation for SigmoidBinaryCrossEntropyLoss from sigmoid (#1345)
    • Fixes jar model url download bug (#1336)
    • Fixes memory in Trainer.checkGradients (#1319)
    • Fixes NDManager is closed bug (#1308)
    • Fixes PyTorch GPU model loading issue (#1302)
    • Fixes MXNet EngineException message (#1300)
    • Fixes python resnet18 demo model GPU bug (#24)
    • Fixes python engine get_as_bytes() bug (#20)

    Contributors

    This release is thanks to the following contributors:

    Source code(tar.gz)
    Source code(zip)
  • v0.13.0(Oct 12, 2021)

    DJL v0.13.0 brings the new TensorRT and Python engines, and updates the engines PyTorch to 1.9.0, ONNXRuntime to 1.9.0, PaddlePaddle to 2.0.2, and introduces several new features:

    Key Features

    • Introduces TensorRT engine
    • Introduces Python engine which allows you to run Python scripts with DJL
    • Upgrades PyTorch engine to 1.9.0 with CUDA 11.1 support
    • Upgrades ONNXRuntime engine to 1.9.0 with UINT8 support
    • Upgrades PaddlePaddle engine to 2.0.2
    • Introduces the djl-bench snap package: sudo snap install djlbench --classic
    • Introduces dynamic batch feature for djl-serving (#1154)
    • DJL serving becomes a standalone repository: https://github.com/deepjavalibrary/djl-serving (#1170)
    • Allows load ModelZoo model using url (#1120)
    • Support .npy and .npz file format (#1131)
    • djl-serving is available in dockerhub
    • Publishes d2l book Chinese translation preview (chapter 1-5) : https://d2l-zh.djl.ai

    Enhancement

    • Introduces several new features in djl-serving:
      • Improves djl-serving API to make it easy to get HTTP headers (#1134)
      • Loads models on all GPUs at startup for djl-serving (#1132)
      • Enables asynchronous logging for djl-serving
      • Makes djl-serving access log in separate log file (#1150)
      • Adds configuration to support number worker threads for GPU inference (#1153)
      • Improves auto-scale algorithm for djl-serving (#1149)
    • Introduces several new features in djl-bench:
      • Adds a command line option to djl-bench to generate NDList file (#1155)
      • Adds warmup to benchmark (#1152)
      • Improves djll-bench to support djl:// urls (#1146)
      • Adds support to benchmark on multiple GPUs (#1144)
      • Adds support to benchmark onnx on GPU machines (#1148)
      • Adds support to benchmark TensorRT models (#1257)
      • Adds support to benchmark Python models (#1267)
    • Introduces several new features in PyTorch engine:
      • Supports PyTorch custom input data type with IValue (#1208)
    • Introduces several new features in OnnxRuntime:
      • Adds UINT8 support for OnnxRuntime (#1271)
    • Introduces several new features in PaddlePaddle:
      • Adds more model loading options for PaddlePaddle (#1173)
      • Adds load functionalities to PaddlePaddle (#1140)
      • Adds remove pass option to PaddlePaddle (#1141)
    • Introduces several API improvements:
      • Adds missing NDList.get(String) API (#1194)
      • Adds support to directly load models from a TFHub url (#1231)
      • Improves repository API to support passing argument in the URL query string (#1139)
      • Avoids loading the default engine if it is not being used (#1136)
      • Improves IO by adding a buffer to read/write (#1135)
      • Improves NDArray.toString() debug mode performance (#1142)
      • Makes GPU device detection engine specific to avoid confusion when using multiple engines (#1138)

    Documentation and examples

    • Adds Style Transfer example with CycleGAN (#1180)

    Breaking change

    • Removes support for Apache MXNet 1.6.0
    • Deprecates Device.getDevices() API - Use Engine.getDevices() instead
    • Renames SimpleVocabulary to DefaultVocabulary

    Bug Fixes

    • Fixes broken link in documents
    • Fixes TensorFlow NDArray was created on CPU instead of GPU bug (#1279)
    • Fixes default image processing pipeline (#1268)
    • Fixed XGBoost NDArray multiple read bug (#1239)
    • Fixes platform matching bug (#1167)
    • Fixes NullPointerException in NDArray.toString() (#1157)
    • Fixes PaddlePaddle crash due to GC (#1162)
    • Fixes NDArrayAdapter.getSparseFormat() unsupported bug (#1151)
    • Fixes mixed device issue in multiple engine use case (#1123)
    • Fixes handle duplicate plugin issue for djl-serving (#1108)
    • Fixes XGBoost NDArray creation bug (#1109)
    • Fixes runtime exception running benchmark arm machine(#1107)
    • Fixes unregister model regression (#1101)

    Contributors

    This release is thanks to the following contributors:

    Source code(tar.gz)
    Source code(zip)
  • v0.12.0(Jul 9, 2021)

    DJL v0.12.0 added GPU support to PaddlePaddle and ONNXRuntime, and introduces several new features:

    Key Features

    • Updates PaddlePaddle engine with GPU support.
    • Updates ONNXRuntime engine with GPU support.
    • Upgrades ONNXRuntime engine to 1.8.0.
    • Upgrades XGBoost engine to 1.4.1.
    • Introduces AWS Inferentia support, see our example for detail.
    • Adds FLOAT16 datatype support in NDArray.
    • Support UTF16 surrogate characters in NLP tokenization.
    • Makes benchmark as a standalone tool.
    • Releases djl-serving docker image to docker hub.

    Enhancement

    • DJL Benchmark now can benchmark any datatype as input.
    • Makes Grayscale image processing match openCV’s behavior (#965)
    • Improves PyTorch engine to load extra shared library for custom operators (#983)
    • Improves djl-serving REST API to support load model on specified engine (#977)
    • Improves djl-serving to support load multiple version of a model on the same endpoint (#1052)
    • Improves djl-serving to support auto-scale workers based on traffic (#986)
    • Implements several operators:
      • Adds the truncated normal operator (#1005)
      • Adds the one hot operator for PyTorch (#1014)
      • Adds the LayerNorm operator in PyTorch (#1069)
    • Introduces several API improvements
      • Improves Criteria.loadModel() API (#1018)
      • Refactors ModleLoader and TranslatorFactory (#712)
      • Improves BlockFactory API (#1045)
      • Makes SpProcessor public API (#1060)

    Documentation and examples

    Breaking change

    • Direct access ModelZoo ModelLoader is no longer supported, use Criteria API instead.
    • Deprecates ModelZoo.loadModel() API in favor of using Criteria.loadModel().

    Bug Fixes

    • Fixes missing softmax in action_recognition model zoo model (#969)
    • Fixes saveModel NPE bug (#989)
    • Fixes NPE bug in block.toString() function (#1076)
    • Adds back String tensor support to TensorFlow engine (lost in 0.11.0 during refactor) (#1040)
    • Sets ai.djl.pytorch.num_interop_threads default value for djl-serving (#1059)

    Known issues

    • The TensorFlow engine has a known memory leak issue due to the JavaCPP dependency. The memory leak issue has been fixed in javacpp 1.5.6-SNAPSHOT. You have to manually include javacpp 1.5.6-SNAPSHOT to avoid the memory leak. See: https://github.com/deepjavalibrary/djl/tree/master/tensorflow/tensorflow-engine#installation for more details.

    Contributors

    This release is thanks to the following contributors:

    • Akshay Rajvanshi(@aksrajvanshi (https://github.com/ghost))
    • Aziz Zayed(@AzizZayed (https://github.com/AzizZayed))
    • Erik Bamberg(@ebamberg (https://github.com/ebamberg))
    • Frank Liu(@frankfliu (https://github.com/frankfliu))
    • Hodovo(@Hodovo (https://github.com/Hodovo))
    • Jake Lee(@stu1130 (https://github.com/stu1130))
    • Qing Lan(@lanking520 (https://github.com/lanking520))
    • Tibor Mezei (@zemei (https://github.com/zemei))
    • Zach Kimberg(@zachgk (https://github.com/zachgk))
    Source code(tar.gz)
    Source code(zip)
  • v0.11.0(May 4, 2021)

    DJL v0.11.0 brings the new engines XGBoost 1.3.1, updates PyTorch to 1.8.1, TensorFlow to 2.4.1, Apache MXNet 1.8.0, PaddlePaddle to 2.0.2 and introduces several new features:

    Key Features

    • Supports XGBoost 1.3.1 engine inference: now you can run prediction using models trained in XGBoost.
    • Upgrades PyTorch to 1.8.1 with CUDA 11.1 support.
    • Upgrades TensorFlow to 2.4.1 with CUDA 11.0 support.
    • Upgrades Apache MXNet to 1.8.0 with CUDA 11.0 support.
    • Upgrades PaddlePaddle to 2.0.2.
    • Upgrades SentencePiece to 0.1.95.
    • Introduces the djl-serving brew package: now you can install djl-serving with brew install djl-serving.
    • Introduces the djl-serving plugins.
    • Introduces Amazon Elastic Inference support.

    Enhancement

    • Improves TensorFlow performance by reducing GC and fixed memory leaking issue (#892)
    • djl-serving now can run all the engines out-of-box (#886)
    • Improves DJL training by using multi-threading on each GPU (#743)
    • Implements several operators:
      • Adds boolean set method to NDArray (#784)
      • Adds batch dot product operator (#849)
      • Adds norm operator to PyTorch (#692)
      • Adds one hot operator (#684)
      • Adds weight decay to Loss (#788)
    • Adds setGraphExecutorOptimize option for PyTorch engine. (#904)
    • Introduces String tensor support for ONNXRuntime (#724)
    • Introduces several API improvements
      • Creates ObjectDetectionDataset (#683)
      • Improves Block usability (#712)
      • Adds BlockFactory feature in model loading (#805)
      • Allows PyTorch stream model loading (#729)
      • Adds NDList decode from InputStream (#734)
      • Adds SymbolBlock Serialization (#687)
    • Introduces model searching feature in djl central (#799)

    Documentation and examples

    Breaking change

    • Renames CheckpointsTrainingListener to SaveModelTrainingListener (#686)
    • Removes erroneous random forest application (#726)
    • Deletes DataManager class (#691)
    • Classes under ai.djl.basicdataset packages has been moved into each sub-packages.

    Bug Fixes

    • Fixes BufferOverflowException when handling handling subimage (#866)
    • Fixes ONNXRuntime 2nd engine dependency from IrisTranslator (#853)
    • Fixes sequenceMask error when n dimension is 2 (#828)
    • Fixes TCP port range buf in djl-serving (#773)
    • Fixes one array case for concat operator (#739)
    • Fixes non-zero operator for PyTorch (#704)

    Known issues

    • TensorFlow engine has known memory leak issue due to JavaCPP dependency. The memory leak issue has been fixed in javacpp 1.5.6-SNAPSHOT. User has to manually include javacpp 1.5.6-SNAPSHOT to avoid memory leak. See: https://github.com/deepjavalibrary/djl/tree/master/tensorflow/tensorflow-engine#installation for more detail.

    Contributors

    This release is thanks to the following contributors:

    • Akshay Rajvanshi(@aksrajvanshi (https://github.com/ghost))
    • Anthony Feenster(@anfee1 (https://github.com/anfee1))
    • Calvin(@mymagicpower (https://github.com/mymagicpower))
    • enpasos(@enpasos (https://github.com/enpasos))
    • Erik Bamberg(@ebamberg (https://github.com/ebamberg))
    • Frank Liu(@frankfliu (https://github.com/frankfliu))
    • G Goswami(@goswamig (https://github.com/goswamig))
    • Hugo Miguel Ferreira(@hmf (https://github.com/hmf))
    • Hodovo(@Hodovo (https://github.com/Hodovo))
    • Jake Lee(@stu1130 (https://github.com/stu1130))
    • Lai Wei(@roywei (https://github.com/roywei))
    • Qing Lan(@lanking520 (https://github.com/lanking520))
    • Marcos(@markbookk (https://github.com/markbookk))
    • Stan Kirdey(@skirdey (https://github.com/skirdey))
    • Zach Kimberg(@zachgk (https://github.com/zachgk))
    • 付颖志 (@fuyz (https://github.com/fuyz))
    • 石晓伟(@Shixiaowei02 (https://github.com/Shixiaowei02))
    Source code(tar.gz)
    Source code(zip)
  • v0.10.0(Feb 24, 2021)

    DJL v0.10.0 brings the new engines PaddlePaddle 2.0 and TFLite 2.4.1, updates PyTorch to 1.7.1, and introduces several new features:

    Key Features

    • Supports PaddlePaddle 2.0 engine inference: now you can run prediction using models trained in PaddlePaddle.

    • Introduces the PaddlePaddle Model Zoo with new models. Please see examples for how to run them.

    • Upgrades TFLite engine to v2.4.1. You can convert TensorFlow SavedModel to TFLite using this converter.

    • Introduces DJL Central to easily browse and view models available in DJL’s ModelZoo.

    • Introduces generic Bert Model in DJL (#105)

    • Upgrades PyTorch to 1.7.1

    Enhancement

    • Enables listing input and output classes in ModelZoo lookup (#624)
    • Improves PyTorch performance by using PyTorch index over engine agnostic solution (#638)
    • Introduces various fixes and improvements for MultiThreadedBenchmark (#617)
    • Makes the default engine deterministic when multiple engines are in dependencies(#603)
    • Adds norm operator (similar to numpy.linalg.norm.html) (#579)
    • Refactors the DJL Trackers to use builder patterns (#562)
    • Adds the NDArray stopGradient and scaleGradient functions (#548)
    • Model Serving now supports scaling up and down (#510)

    Documentation and examples

    • Introduces DJL 101 Video Series on DJL Youtube Channel
    • Adds the documentation for Applications (#673)
    • Adds an introduction for Engines (#660)
    • Adds documents for DJL community and forums (#646)
    • Adds documents on community leaders (#572)
    • Adds more purpose to the block tutorial and miscellaneous docs (#607)
    • Adds a DJL Paddle OCR Example (#568)
    • Adds a TensorFlow amazon review Jupyter notebook example

    Breaking change

    • Renames DJL-Easy to DJL-Zero (#519)
    • Makes RNN operators generic across engines (#554)
    • Renames CheckpointsTrainingListener to SaveModelTrainingListener (#573)
    • Makes Initialization optional in training (#533)
    • Makes SoftmaxCrossEntropyLoss's fromLogit flag mean inputs are un-normalized (#639)
    • Refactors the Vocabulary builder API
    • Refactors the SymbolBlock with AbstractSymbolBlock (#491)

    Bug Fixes

    • Fixes the benchmark rss value #656
    • Fixes the recurrent block memory leak and the output shape calculation (#556)
    • Fixes the NDArray slice size (#550)
    • Fixes #493: verify-java plugin charset bug (#496)
    • Fixes #484: support arbitrary URL scheme for repository
    • Fixes AbstractBlock inputNames and the inputShapes mismatch bug

    Known issues

    • The training tests fail on GPU and Windows CPU if 3 engines(MXNet, PyTorch, TensorFlow) are loaded and run together

    Contributors

    This release is thanks to the following contributors:

    Source code(tar.gz)
    Source code(zip)
  • v0.9.0(Dec 18, 2020)

    DJL 0.9.0 brings MXNet inference optimization, abundant PyTorch new feature support, TensorFlow windows GPU support and experimental DLR engine that support TVM models.

    Key Features

    • Add experimental DLR engine support. Now you can run TVM model with DJL

    MXNet

    • Improve MXNet JNA layer by reusing String, String[] and PointerArray with object pool which reduce the GC time significantly

    PyTorch

    • you can easily create COO Sparse Tensor with following code snippet
    long[][] indices = {{0, 1, 1}, {2, 0, 2}};
    float[] values = {3, 4, 5};
    FloatBuffer buf = FloatBuffer.wrap(values);
    manager.createCoo(FloatBuffer.wrap(values), indices, new Shape(2, 4));
    
    • If the input of your TorchScript model need List or Dict type, we now add simple one dimension support for you.
    // assum your torchscript model takes model({'input': input_tensor})
    // you tell us this kind of information by setting the name
    NDArray array = manager.ones(new Shape(2, 2));
    array.setName("input1.input");
    
    • we support loading ExtraFilesMap
    // saving ExtraFilesMap
    Criteria<Image, Classifications> criteria = Criteria.builder()
      ...
      .optOption("extraFiles.dataOpts", "your value")  // <- pass in here 
      ... 
    

    TensorFlow

    • Windows GPU is now supported

    Several Engines upgrade

    | Engine | version | | ----------- | ------- | | PyTorch | 1.7.0 | | TensorFlow | 2.3.1 | | fastText | 0.9.2 |

    Enhancement

    • Add docker file for serving
    • Add Deconvolution support for MXNet engine
    • Support PyTorch COO Sparse tensor
    • Add CSVDataset, you can find a sample usage here
    • Upgrade TensorFlow to 2.3.1
    • Upgrade PyTorch to 1.7.0
    • Add randomInteger operator support for MXNet and PyTorch engine
    • Add PyTorch Profiler
    • Add TensorFlow Windows GPU support
    • Support loading the model from jar file
    • Support 1-D list and dict input for TorchScript
    • Remove the Pointer class being used for JNI to relieve Garbage Collector pressure
    • Combine several BertVocabulary into one Vocabulary
    • Add loading the model from Path class
    • Support ExtraFilesMap for PyTorch model inference
    • Allow both int32 & int64 for prediction & labels in TopKAccuracy
    • Refactor MXNet JNA binding to reduce GC time
    • Improve PtNDArray set method to use ByteBuffer directly and avoid copy during tensor creation
    • Support experimental MXNet optimizeFor method for accelerator plugin.

    Documentation and examples

    • Add Amazon Review Ranking Classification
    • Add Scala Spark example code on Jupyter Notebook
    • Add Amazon SageMaker Notebook and EMR 6.2.0 examples
    • Add DJL benchmark instruction

    Bug Fixes

    • Fix PyTorch Android NDIndex issue
    • Fix Apache NiFi issue when loading multiple native in the same Java process
    • Fix TrainTicTacToe not training issue
    • Fix Sentiment Analysis training example and FixedBucketSampler
    • Fix NDArray from DataIterable not being attaching to NDManager properly
    • Fix WordPieceTokenizer infinite loop
    • Fix randomSplit dataset bug
    • Fix convolution and deconvolution output shape calculations

    Contributors

    Thank you to the following community members for contributing to this release:

    Frank Liu(@frankfliu) Lanking(@lanking520) Kimi MA(@kimim) Lai Wei(@roywei) Jake Lee(@stu1130) Zach Kimberg(@zachgk) 0xflotus(@0xflotus) Joshua(@euromutt) mpskowron(@mpskowron) Thomas(@thhart) DocRozza(@docrozza) Wai Wang(@waicool20) Trijeet Modak(@uniquetrij)

    Source code(tar.gz)
    Source code(zip)
  • v0.8.0(Sep 23, 2020)

    DJL 0.8.0 is a release closely following 0.7.0 to fix a few key bugs along with some new features.

    Key Features

    • Search model zoo with criteria
    • Standard BERT transformer and WordpieceTokenizer for more BERT tasks
    • Simplify MRL and Remove Anchor
    • Simplify and Standardize CV Model
    • Improve Model describe input and output
    • String NDArray support (only for TensorFlow Engine)
    • Add erfinv operator support
    • MXNet 1.6.0 backward compatibility, now you can switch MXNet versions (1.6 and 1.7) using DJL 0.8.0
    • Combined pytorch-engine-precxx-11 and pytorch-engine package
    • Upgrade onnx runtime from 1.3.1 to 1.4.0

    Documentation and examples

    • Object Detection with TensorFlow saved model example
    • Text Classification with TensorFlow BERT model example
    • Added more documentation on TensorFlow engine.

    Bug Fixes

    • Fixed MXNet multithreading bug and updated multi-threading documentation
    • Fixed TensorFlow 2.3 native binaries for Windows platform

    Known issues

    • You need to add your own Translator when loading image classification models(ResNet, MobileNet) from TensorFlow model Zoo, refer to the example here.

    Contributors

    Thank you to the following community members for contributing to this release:

    Dennis Kieselhorst, Frank Liu, Jake Cheng-Che Lee, Lai Wei, Qing Lan, Zach Kimberg, uniquetrij

    Source code(tar.gz)
    Source code(zip)
  • v0.7.0(Sep 4, 2020)

    DJL 0.7.0 brings SetencePiece for tokenization, GravalVM support for PyTorch engine, a new set of Nerual Network operators, BOM module, Reinforcement Learning interface and experimental DJL Serving module.

    Key Features

    • Now you can leverage powerful SentencePiece to do text processing including tokenization, de-tokenization, encoding and decoding. You can find more details on extension/sentencepiece.
    • Engine upgrade:
      • MXNet engine: 1.7.0-backport
      • PyTorch engine: 1.6.0
      • TensorFlow: 2.3.0
    • MXNet multi-gpu training now is boosted by MXNet KVStore by default, which saves lots of overhead by GPU memory copy.
    • GraalVM are fully supported on both of regular execution and native image for PyTorch engine. You can find more details on GraalVM example.
    • Add a new set of Neural Network operators that offers capability of full controlling over parameters for CV domain, which is similar to PyTorch nn.functional module. You can find the operator method in its Block class.
    Conv2d.conv2d(NDArray input, NDArray weight, NDArray bias, Shape stride, Shape padding, Shape dilation, int groups);
    
    • Bill of Materials (BOM) is introduced to manage the version of dependencies for you. In DJL, the engine you are using usually is tied to a specific version of native package. By easily adding BOM dependencies like this, you won’t worry about version anymore.
    <dependency>
        <groupId>ai.djl</groupId>
        <artifactId>bom</artifactId>
        <version>0.7.0</version>
        <type>pom</type>
        <scope>import</scope>
    </dependency>
    
    implementation platform("ai.djl:bom:0.7.0")
    
    • JDK 14 now get supported
    • New Reinforcement Learning interface including RIAgent, RlEnv, etc, you can see a comprehensive TicTacToe example.
    • Support DJL Serving module. With only a single command, now you can enjoy deploying your model without bothering writing the server code or config like server proxy.
    cd serving && ./gradlew run --args="-m https://djl-ai.s3.amazonaws.com/resources/test-models/mlp.tar.gz"
    

    Documentation and examples

    • We wrote the D2L book from chapter 1 to chapter 7 with DJL. You can learn basic deep learning concept and classic CV model architecture with DJL. Repo
    • We launched a new doc website that hosts abundant documents and tutorials for quick search and copy-paste.
    • New Online Sentiment Analysis with Apache Flink.
    • New CTR prediction using Apache Beam and Deep Java Library(DJL).
    • New DJL logging configuration document which includes how to enable slf4j, switch to other logging libraries and adjust log level to debug the DJL.
    • New Dependency Management document that lists DJL internal and external dependencies along with their versions.
    • New CV Utilities document as a tutorial for Image API.
    • New Cache Management document is updated with more detail on different categories.dependency management.
    • Update Model Loading document to describe loading model from various sources like s3, hdfs.

    Enhancement

    • Add archive file support to SimpleRepository
    • ImageFolder supports nested folder
    • Add singleton method for LambdaBlock to avoid redundant function reference
    • Add Constant Initializer
    • Add RMSProp, Adagrad, Adadelta Optimizer for MXNet engine
    • Add new tabular dataset: Airfoil Dataset
    • Add new basic dataset: CookingExchange, BananaDetection
    • Add new NumPy like operators: full, sign
    • Make prepare() method in Dataset optional
    • Add new Image augmentation APIs where you can add to Pipeline to enrich your image dataset
    • Add new handy fromNDArray to Image API for converting NDArray to Image object quickly
    • Add interpolation option for Image Resize operator
    • Support archive file for s3 repository
    • Import new SSD model from TensorFlow Hub into DJL model zoo
    • Import new Sentiment Analysis model from HuggingFace into DJL model zoo

    Breaking changes

    • Drop CUDA 9.2 support for all the platforms including linux, windows
    • The arguments of several blocks are changed to align with the signature of other widely used Deep Learning frameworks, please refer to our Java doc site
    • FastText is no longer a full Engine, it becomes a part of NLP utilities in favor of FastTextWorkEmbedding
    • Move the WarmUp out from existing Tracking and introduce new WarmUpTracker
    • MxPredictor now doesn’t copy parameters by default, please make sure to use NaiveEngine when you run inference in multi-threading environment

    Bug Fixes

    • Fixing Validation Epoch Result bug
    • Fix multiple process downloading the same model bug
    • Fix potential concurrent write bug while downloading metadata.json
    • Fix URI parsing error on Windows
    • Fix multi-gpu training crash when the number of the batch size is smaller than number of devices
    • Fix not setting number of inter-op threads for PyTorch engine

    Contributors

    Thank you to the following community members for contributing to this release:

    Christoph Henkelmann, Frank Liu, Jake Cheng-Che Lee, Jake Lee, Keerthan Vasist, Lai Wei, Qing Lan, Victor Zhu, Zach Kimberg, aksrajvanshi, gstu1130, 蔡舒起

    Source code(tar.gz)
    Source code(zip)
  • v0.6.0(Jun 25, 2020)

    DJL 0.6.0 brings stable Android support, ONNX Runtime experimental inference support, experimental training support for PyTorch.

    Key Features

    • Stable Android inference support for PyTorch models
      • Provide abstraction for Image processing using ImageFactory
    • Experimental support for inference on ONNX models
    • Initial experimental training and imperative inference support for PyTorch engine
    • Experimental support for using multi-engine
    • Improved usage for NDIndex - support for ellipsis notation, arguments
    • Improvements to AbstractBlock to simplify custom block creation
    • Added new datasets

    Documentation and examples

    Breaking changes

    • ModelZoo Configuration changes
    • ImageFactory changes
    • Please refer to javadocs for minor API changes

    Known issues

    • Issue with training with MXNet in multi-gpu instances

    Contributors

    Thank you to the following community members for contributing to this release:

    Christoph Henkelmann, Frank Liu, Jake Lee, JonTanS, Keerthan Vasist, Lai Wei, Qing, Qing Lan, Victor Zhu, Zach Kimberg, ai4java, aksrajvanshi

    Source code(tar.gz)
    Source code(zip)
  • v0.5.0(May 12, 2020)

    DJL 0.5.0 release brings TensorFlow engine inference, initial NLP support and experimental Android inference with PyTorch engine.

    Key Features

    • TensorFlow engine support with TensorFlow 2.1.0
      • Support NDArray operations, TensorFlow model zoo, multi-threaded inference
    • PyTorch engine improvement with PyTorch 1.5.0
    • Experimental Android Support with PyTorch engine
    • MXNet engine improvement with MXNet 1.7.0
    • Initial NLP support with MXNet engine
      • Training LSTM models
      • Support various text/word embedding, Seq2Seq use cases
      • Added NLP datasets
    • New AWS-AI toolkit to integrate with AWS technologies
      • Load model from s3 buckets directly
    • Improved model-zoo with more models

    Documentation and examples

    Breaking changes

    • We moved our repository module under api module. There will be no 0.5.0 version for ai.djl.repository, use ai.djl.api instead.
    • Please refer to DJL Java Doc for some minor API changes.

    Know issues:

    • Issue when using multiple Engines at the same time: https://github.com/awslabs/djl/issues/57
    • Issue using DJL with Quarkus: https://github.com/awslabs/djl/issues/67
    • We saw random crash on mac for transfer Learning on CIFAR-10 Dataset example on Jupyter Notebook. Command line all works.
    Source code(tar.gz)
    Source code(zip)
  • v0.4.1(Apr 6, 2020)

    DJL 0.4.1 release includes an important performance Improvement on MXNet engine:

    Performance Improvement:

    • Cached MXNet features. This will avoid MxNDManager.newSubManager() to repeatedly calling getFeature() which will make JNA calls to native code.

    Known Issues:

    Same as v0.4.0 release:

    • PyTorch engine doesn't fully support multithreaded inference. You may see random crashes. Single-threaded inference is not impacted. We expect to fix this issue in a future release.
    • We saw random crash on mac for “transfer Learning on CIFAR-10 Dataset” example on Jupyter Notebook. Command line all works.
    Source code(tar.gz)
    Source code(zip)
  • v0.4.0(Mar 30, 2020)

    DJL 0.4.0 brings PyTorch and TensorFlow 2.0 inference support. Now you can use these engines directly from DJL with minimum code changes.

    Note: TensorFlow 2.0 currently is in PoC stage, users will have to build from source to use it. We expect TF Engine finish in the future releases.

    Key Features

    • Training improvement
      • Add InputStreamTranslator
    • Model Zoo improvement
      • Add LocalZooProvider
      • Add ListModels API
    • PyTorch Engine support
      • Use the new ai.djl.pytorch:pytorch-native-auto dependency for automatic engine selection and a simpler build/installation process
      • 60+ methods supported
    • PyTorch ModelZoo support
      • Image Classification models: ResNet18 and ResNet50
      • Object Detection model: SSD_ResNet50
    • TensorFlow 2.0 Engine support
      • Support on Eager Execution for imperative mode
      • 30+ methods support
    • TensorFlow ModelZoo support
      • Image Classification models: ResNet50, MobileNetV2

    Breaking Changes

    There are a few changes in API and ModelZoo packages to adapt to multi-engine support. Please follow our latest examples to update your code base from 0.3.0 to 0.4.0.

    Known Issues

    1. PyTorch engine doesn't fully support multithreaded inference. You may see random crashes. Single-threaded inference is not impacted. We expect to fix this issue in a future release.
    2. We saw random crash on mac for “transfer Learning on CIFAR-10 Dataset” example on Jupyter Notebook. Command line all works.
    Source code(tar.gz)
    Source code(zip)
  • v0.3.0(Feb 24, 2020)

    This is the v0.3.0 release of DJL

    Key Features

    • Use the new ai.djl.mxnet:mxnet-native-auto dependency for automatic engine selection and a simpler build/installation process
    • New Jupyter Notebook based tutorial for DJL
    • New Engine Support for:
      • FastText Engine
      • Started implementation on a PyTorch Engine
    • Simplified training experience featuring:
      • TrainingListeners to easily provide full featured training
      • DefaultTrainingConfig now contains a default optimizer and initializer
      • Easier to transfer from examples to your own code
    • Specify the random seed for reproducible training
    • Run with multiple engines and specify the default using the "DJL_DEFAULT_ENGINE" environment variable or "ai.djl.default_engine" system property
    • Updated ModelZoo design to support unified loading with Criteria
    • Simple random Hyperparameter optimization

    Breaking Changes

    DJL is working to further improve the ease of use and correctness of our API. To that end, we have made a number of breaking changes for this release. Here are a few of the areas that had breaking changes:

    • Renamed TrainingMetrics to Evaluator
    • CompositeLoss replaced with AbstractCompositeLoss and SimpleCompositeLoss
    • Modified MLP class
    • Remove Matrix class
    • Updates to NDArray class

    Known Issues

    1. RNN operators do not working with GPU on Windows.
    2. Only CUDA_ARCH 37, 70 are supported for Windows GPU machine.
    Source code(tar.gz)
    Source code(zip)
  • v0.2.1(Dec 18, 2019)

    This is the v0.2.1 release of DJL

    Key Features

    1. Added support for Windows 10.
    2. CUDA 9.2 support for all supported Operating systems (Linux, Windows)

    Known Issues

    1. RNN operators do not working with GPU on Windows.
    2. Only CUDA_ARCH 37, 70 are supported for Windows GPU machine.
    Source code(tar.gz)
    Source code(zip)
  • v0.2.0(Nov 29, 2019)

    This is the v0.2.0 release for DJL Key Features

    1. Deep learning engine agnostic High-level API for Training and Prediction.
    2. Dataset API, to create objects based out of different dataset formats, that work with training batchifier.
    3. MXNet-Model-Zoo with pre-trained models for Image Classification, Object Detection, Segmentation and more.
    4. Jupyter Notebooks and Examples to help get started with Training and predicting models with DJL

    Engines currently supported

    1.Apache MXNet

    Javadocs

    The javadocs are available here

    Source code(tar.gz)
    Source code(zip)
Owner
Amazon Web Services - Labs
AWS Labs
Amazon Web Services - Labs
java deep learning algorithms and deep neural networks with gpu acceleration

Deep Neural Networks with GPU support Update This is a newer version of the framework, that I developed while working at ExB Research. Currently, you

Ivan Vasilev 1.2k Jan 6, 2023
On-device wake word detection powered by deep learning.

Porcupine Made in Vancouver, Canada by Picovoice Porcupine is a highly-accurate and lightweight wake word engine. It enables building always-listening

Picovoice 2.8k Dec 30, 2022
Datumbox is an open-source Machine Learning framework written in Java which allows the rapid development of Machine Learning and Statistical applications.

Datumbox Machine Learning Framework The Datumbox Machine Learning Framework is an open-source framework written in Java which allows the rapid develop

Vasilis Vryniotis 1.1k Dec 9, 2022
Statistical Machine Intelligence & Learning Engine

Smile Smile (Statistical Machine Intelligence and Learning Engine) is a fast and comprehensive machine learning, NLP, linear algebra, graph, interpola

Haifeng Li 5.7k Jan 1, 2023
Java Statistical Analysis Tool, a Java library for Machine Learning

Java Statistical Analysis Tool JSAT is a library for quickly getting started with Machine Learning problems. It is developed in my free time, and made

null 752 Dec 20, 2022
Learning Based Java (LBJava)

Learning Based Java LBJava core LBJava examples LBJava maven plugin Compiling the whole package From the root directory run the following command: Jus

CogComp 12 Jun 9, 2019
Tribuo - A Java machine learning library

Tribuo - A Java prediction library (v4.2) Tribuo is a machine learning library in Java that provides multi-class classification, regression, clusterin

Oracle 1.1k Dec 28, 2022
Java time series machine learning tools in a Weka compatible toolkit

UEA Time Series Classification A Weka-compatible Java toolbox for time series classification, clustering and transformation. For the python sklearn-co

Machine Learning and Time Series Tools and Datasets 140 Nov 7, 2022
MALLET is a Java-based package for statistical natural language processing, document classification, clustering, topic modeling, information extraction, and other machine learning applications to text.

MALLET is a Java-based package for statistical natural language processing, document classification, clustering, topic modeling, information extraction, and other machine learning applications to text.

null 900 Jan 2, 2023
Oryx 2: Lambda architecture on Apache Spark, Apache Kafka for real-time large scale machine learning

Oryx 2 is a realization of the lambda architecture built on Apache Spark and Apache Kafka, but with specialization for real-time large scale machine l

Oryx Project 1.8k Dec 28, 2022
A machine learning package built for humans.

aerosolve Machine learning for humans. What is it? A machine learning library designed from the ground up to be human friendly. It is different from o

Airbnb 4.8k Dec 30, 2022
statistics, data mining and machine learning toolbox

Disambiguation (Italian dictionary) Field of turnips. It is also a place where there is confusion, where tricks and sims are plotted. (Computer scienc

Aurelian Tutuianu 63 Jun 11, 2022
Oryx 2: Lambda architecture on Apache Spark, Apache Kafka for real-time large scale machine learning

Oryx 2 is a realization of the lambda architecture built on Apache Spark and Apache Kafka, but with specialization for real-time large scale machine l

Oryx Project 1.7k Mar 12, 2021
Test project for learning GoF design pattern

DesignPattern Test project for learning GoF design pattern ㅁ개요 객체지향 설계의 교과서라고 불리는 Design Pattern 을 직접 Activity 별로 구현해봤습니다. ㅁ동기 물론 디자인패턴을 몰라도 기능은 얼마든지

null 11 Aug 8, 2022
A course for learning how to program FRC robots using the WPILib and a Romi robot.

FRC-Romi-Programming-Course A course for learning how to program FRC robots using the WPILib and a Romi robot. This course is designed for FRC teams o

null 16 Nov 9, 2022
Bazel training materials and codelabs focused on beginner, advanced and contributor learning paths

Bazel-learning-paths This repo has materials for learning Bazel: codelabs, presentations, examples. We are open sourcing the content for training engi

null 18 Nov 14, 2022
Reference implementation for MINAS (MultI-class learNing Algorithm for data Streams), an algorithm to address novelty detection in data streams multi-class problems.

Reference implementation for MINAS (MultI-class learNing Algorithm for data Streams), an algorithm to address novelty detection in data streams multi-class problems.

Douglas M. Cavalcanti 4 Sep 7, 2022