A few lines of code, easy to achieve target detection and image classification tasks — build enterprise level machine learning micro service based on spring boot and djl

Time:2021-3-3

A few lines of code, easy to achieve target detection and image classification tasks -- build enterprise level machine learning micro service based on spring boot and djl

Now there seems to be a trend in AI applications, and most of the learning applications are in Python. Spring boot is widely used as an open source platform in the field of micro services. Users can use stock API to create an RPC API based on C + + or Python applications to achieve reasoning tasks in different languages. But it will bring a lot of maintenance costs and efficiency problems. For RPC communication alone, the time consumption of communication alone has reached the time consumption of reasoning, which has become the bottleneck of improving the overall application speed. Now, AWS cloud service has launched an open source Java library based on deep learning, deep Java library (djl), which simplifies the cumbersome development process and brings better and more stable solutions to the majority of developers.

preface

Many users of AWS cloud services, whether start-ups or large companies, are gradually carrying machine learning (ML) and deep learning (DL) tasks on top of their existing products. A large number of machine learning tasks are applied in commercial and industrial fields, such as target detection in images and videos, sentiment analysis in documents, fraud detection in big data. Although machine learning is widely used in other languages (such as Python), the cost of learning and integration is still very high for developers of existing products. Just imagine, if you want to use another language to integrate into existing Java services, you have to make a lot of changes from code writing, compilation, testing to final deployment. In order to solve this problem, this article will propose a new way to solve the problem: users can directly deploy machine learning applications in existing services without re deploying existing resources and personnel.

Application of spring boot in production environment

Spring boot is an open source platform widely used in the field of micro services. Its main feature is to simplify the process of distributed system distribution management application. But at present, there are only a few options for users to deploy ml applications. Take reasoning application for example, users can use stock API to create an RPC API based on C + + or Python application to realize reasoning tasks in different languages. Although this solution can solve the urgent need of deployment in the shortest time, it has caused a lot of maintenance costs and efficiency problems from the long-term running effect. As far as RPC communication is concerned, the time consumption of communication alone may have reached the time consumption of reasoning, which makes this solution become the bottleneck of improving the overall application speed.

In order to better solve the dilemma of developers, AWS cloud service has launched an open source Java library based on deep learning,DeepJavaLibrary (DJL). Djl’s service aim is to simplify the expensive and cumbersome development process and bring better and more stable solutions to the majority of developers. This article will start from a basic spring boot application, using theDeep Java Library (DJL)To integrate machine learning applications in micro services. With only a few lines of code, the task of object detection and image classification can be easily realized.

Configure spring boot starter (SBS)

Spring boot starter is a one-stop spring library management tool. It simplifies many operations needed to reference new libraries, such as copying and pasting sample code, modifying configuration files, etc. Please refer to spring boot starterOfficial GuideFor more information. In our teaching today, we will useDJL Spring Boot StarterAn SBS with deep learning deployment function added. On the basis of the existing architecture, djl SBS addsAutomatic configurationThe function of. It allows users to use dependencies as beans without worrying about them in a few lines of code. If any of the following steps are not very clear, you can refer to usSample application

Dependency management

Djl library can be applied to various operating system platforms, and also supports a variety of deep learning engines, such as tensorflow 2.0, pytorch and mxnet. Djl built in a series of automatic selection mechanism, users do not need to choose the operating system to run. But djl still needs users to choose one or more deep learning engines. Taking mxnet as an example, users can choose the following configuration( pom.xml ):

<parent>
  <artifactId>spring-boot-starter-parent</artifactId>
  <groupId>org.springframework.boot</groupId>
  <version>2.2.6.RELEASE</version>
</parent>

<properties>
  < java.version >11</ java.version ><! -- 11 is the minimum supported version of Java -- >
  < jna.version >5.3.0</ jna.version ><! -- need to override JNA version -- >
</properties> 

<dependency>
  <groupId>ai.djl.spring</groupId>
  <artifactId>djl-spring-boot-starter-mxnet-linux-x86_64</artifactId>
  <version>${djl.starter.version}</version> <!-- e.g. 0.2 -->
</dependency>

Users can choose the platform they want to run on<dependency>In the middle,linux-x86_64(Linux) can be replaced bywin-x86_64(Windows), orosx-x86_64 (Mac OS)。

We also provide a way to automatically find the corresponding system dependencies at run timeauto
`<dependency>
<groupId>ai.djl.spring</groupId>
<artifactId>djl-spring-boot-starter-mxnet-auto</artifactId>
<version>${djl.starter.version}</version> <!– e.g. 0.2 –>
</dependency>`

If you need to use pytorch, you can make the following changes:

<dependency>
  <groupId>ai.djl.spring</groupId>
  <artifactId>djl-spring-boot-starter-pytorch-auto</artifactId>
  <version>${djl.starter.version}</version> <!-- e.g. 0.2 and above -->
</dependency>

The configuration of gradle is also very similar. Only the following lines are needed:

plugins {
  ...
  id("org.springframework.boot")
}
repositories {
  Maven central() // packages published in Maven Central
}

dependencies {
  implementation("ai.djl.spring:djl-spring-boot-starter-mxnet-auto:0.2")
}

Note that since springboot itself uses an old version of the JNA library, we need to set it manuallygradle.propertiesInside"jna.version=5.3.0"

Using spring’s automatic selection function

Next, we can use spring’s auto configuration to achieve automatic selection by adding the following dependencies:

<dependency>
  <groupId>ai.djl.spring</groupId>
  <artifactId>djl-spring-boot-starter-autoconfigure</artifactId>
  <version>${djl.starter.version}</version>
</dependency>

In gradle:

dependencies {
  implementation("ai.djl.spring:djl-spring-boot-starter-autoconfigure:${djl.starter.version}")
}

After importing the dependency, spring boot will automatically configure the environment and find the model. Developers need to provide a standard spring configuration file, such asapplication.ymlperhapsapplication.properties. The model type can be any of the following:

 QUESTION_ANSWER(NLP.QUESTION_ANSWER),
  TEXT_CLASSIFICATION(NLP.TEXT_CLASSIFICATION),
  IMAGE_CLASSIFICATION(CV.IMAGE_CLASSIFICATION),
  OBJECT_DETECTION(CV.OBJECT_DETECTION),
  ACTION_RECOGNITION(CV.ACTION_RECOGNITION),
  INSTANCE_SEGMENTATION(CV.INSTANCE_SEGMENTATION),
  POSE_ESTIMATION(CV.POSE_ESTIMATION),
  SEMANTIC_SEGMENTATION(CV.SEMANTIC_SEGMENTATION);

For example, if you want to do target detection, you can chooseOBJECT_DETECTION. You can refer to the followingyamlTo configure:

djl:
    #Set application type
    application-type: OBJECT_DETECTION
    #Set input data format. Some models support multiple data formats
    input-class: java.awt.image.BufferedImage
    #Setting output data format
    output-class: ai.djl.modality.cv.output.DetectedObjects
    #Set a filter to filter your model
    model-filter:
      size: 512
      backbone: mobilenet1.0
    #Override existing I / O configuration
    arguments:
      Threshold: 0.5

IDE support

We recommend that users use IDE, such as IntelliJ or eclipse

Image URL: https://github.com/awslabs/djl-spring-boot-starter-demo/raw/master/docs/media/djl-start-ide-support-low-frame-30s.gif

IntelliJ can use
Ctrl+SpaceTo do it automatically
Ctrl+JTo quickly query documents

Run your app

Now let’s try the effect of the previous configuration. From now on, we only need two steps to complete all the model deployment and running tasks. Before that, developers only need to create a simple single class spring application.

Step 1: inject predictor to do target detection

 @Resource 
 private Supplier<Predictor> predictorProvider;

Step 2: run target detection

try (var predictor = predictorProvider.get()) {
    var results = predictor.predict(ImageIO.read(this.getClass()
          .getResourceAsStream("/puppy-in-white-and-red-polka.jpg")));

    for(var result : results.items()) {
        LOG.info("results: {}", result.toString());
    }
}

If you use our example, the following results will be displayed in the console:

a.d.s.e.console.ConsoleApplication: results: class: "dog", probability: 0.90820, bounds: {x=0.487, y=0.057, width=0.425, height=0.484}

Quick reproduction

You can easily run the sample application with just a few lines

git clone [email protected]:awslabs/djl-spring-boot-starter.git
cd djl-spring-boot-starter/djl-spring-boot-console-sample 
../mvnw package 
../mvnw spring-boot:run

We have also provided oneMore complicated examples, you can use a variety of plug-ins to quickly implement a restful classifier microservice.

Understanding djl

A few lines of code, easy to achieve target detection and image classification tasks -- build enterprise level machine learning micro service based on spring boot and djl
DJLIt is a deep learning framework tailored for Java developers launched by AWS cloud service at re: invest conference in 2019, which has been running in millions of reasoning tasks of Amazon. If we want to summarize the main features of djl, there are three points as follows:

Djl is not limited to back-end engine: users can easily use mxnet, pytorch, tensorflow and fasttext to do model training and reasoning on Java.
Djl’s operator design is infinitely close to numpy: its use experience is basically seamless with numpy, and switching engines will not change the result.
Djl excellent memory management and efficiency mechanism: djl has its own resource recovery mechanism, 100 hours of continuous reasoning will not overflow memory.

Now you can run djl on MAC / Linux / windows. Djl has the function of self checking CUDA version, and will automatically use the corresponding CUDA version package to run GPU tasks. For more information, see the following links:

https://djl.ai
https://github.com/awslabs/djl
Welcome to djl, tooSlack Forum

A few lines of code, easy to achieve target detection and image classification tasks -- build enterprise level machine learning micro service based on spring boot and djl