20 lines of code: image classification and prediction can be easily done in Python under serverless architecture

Time:2021-8-2

20 lines of code: image classification and prediction can be easily done in Python under serverless architecture

Author Jiang Yu

preface

Image classification is a hot topic in the field of artificial intelligence. Popular interpretation is an image processing method that distinguishes different types of targets according to their different characteristics reflected in the image information.

It uses computer to quantitatively analyze the image, and classifies each pixel or region in the image into one of several categories to replace human visual interpretation.

Image classification is often encountered in actual production and life, and has strong pertinence to different fields or needs. For example, flower information is recognized by photographing flowers, and person information is compared by face.

Usually, these image recognition or classification tools collect data at the client and calculate at the server to obtain the results, that is, generally, there are special APIs to realize image recognition. For example, major cloud manufacturers will provide us with similar capabilities for a fee:

Alibaba cloud image recognition page:

20 lines of code: image classification and prediction can be easily done in Python under serverless architecture

Huawei cloud image recognition page:

20 lines of code: image classification and prediction can be easily done in Python under serverless architecture

Through an interesting Python library, this paper will quickly build the image classification function on the cloud function, and combine it with the API gateway to provide API functions and realize a serverless architecture“Image classification API”

First, let’s introduce the required dependency libraries:ImageAI。 Through the official documents of this dependency, we can see the following description:

Imageai is a python library designed to enable developers to use a few simple lines of code to build applications and systems with deep learning and computer vision capabilities.
Based on the principle of simplicity, imageai supports the most advanced machine learning algorithms for image prediction, user-defined image prediction, object detection, video detection, video object tracking and image prediction training. Imageai currently supports image prediction and training using four different machine learning algorithms trained on imagenet-1000 data sets. Imageai also supports object detection, video detection and object tracking using retinanet trained on coco data sets. Finally, imageai will provide broader and more professional support for computer vision, including but not limited to image recognition in special environments and special fields.

In other words, this dependency library can help us complete basic image recognition and video target extraction. Although it gives some data sets and models, we can also carry out additional training and customized expansion according to our own needs. Through the official code, we can see a simple demo:

# -*- coding: utf-8 -*-
from imageai.Prediction import ImagePrediction

#Model loading
prediction = ImagePrediction()
prediction.setModelTypeAsResNet()
prediction.setModelPath("resnet50_weights_tf_dim_ordering_tf_kernels.h5")
prediction.loadModel()

predictions, probabilities = prediction.predictImage("./picture.jpg", result_count=5 )
for eachPrediction, eachProbability in zip(predictions, probabilities):
    print(str(eachPrediction) + " : " + str(eachProbability))

When we specify picture.jpg as:

20 lines of code: image classification and prediction can be easily done in Python under serverless architecture

Our results after implementation are:

laptop : 71.43893241882324
notebook : 16.265612840652466
modem : 4.899394512176514
hard_disc : 4.007557779550552
mouse : 1.2981942854821682

If you feel that the model resnet50_ weights_ tf_ dim_ ordering_ tf_ Kernels.h5 is too large and takes too long. You can select the model according to your needs:

  • Squeezenet (file size: 4.82 MB, shortest prediction time, moderate accuracy)
  • Resnet50 by Microsoft Research (file size: 98 MB, fast prediction time and high accuracy)
  • Inception V3 by Google brain team (file size: 91.6 MB, slow prediction time and higher accuracy)
  • Densenet121 by Facebook AI research (file size: 31.6 MB, slow prediction time, highest accuracy)

For the model download address, please refer to GitHub address:
https://github.com/OlafenwaMoses/ImageAI/releases/tag/1.0

Or refer to the official imageai documentation:
https://imageai-cn.readthedocs.io/zh_CN/latest/ImageAI_Image_Prediction.html

Project serverless

Write the entry method and initialize the project according to the requirements of function calculation. At the same time, create the folder model under the current project and copy the model file to this folder:

20 lines of code: image classification and prediction can be easily done in Python under serverless architecture

Overall process of the project:

20 lines of code: image classification and prediction can be easily done in Python under serverless architecture

Implementation code:

# -*- coding: utf-8 -*-

from imageai.Prediction import ImagePrediction
import json
import uuid
import base64
import random


# Response
class Response:
    def __init__(self, start_response, response, errorCode=None):
        self.start = start_response
        responseBody = {
            'Error': {"Code": errorCode, "Message": response},
        } if errorCode else {
            'Response': response
        }
        #UUID is added by default to facilitate later positioning
        responseBody['ResponseId'] = str(uuid.uuid1())
        print("Response: ", json.dumps(responseBody))
        self.response = json.dumps(responseBody)

    def __iter__(self):
        status = '200'
        response_headers = [('Content-type', 'application/json; charset=UTF-8')]
        self.start(status, response_headers)
        yield self.response.encode("utf-8")


#Random string
randomStr = lambda num=5: "".join(random.sample('abcdefghijklmnopqrstuvwxyz', num))

#Model loading
print("Init model")
prediction = ImagePrediction()
prediction.setModelTypeAsResNet()
print("Load model")
prediction.setModelPath("/mnt/auto/model/resnet50_weights_tf_dim_ordering_tf_kernels.h5")
prediction.loadModel()
print("Load complete")


def handler(environ, start_response):
    try:
        request_body_size = int(environ.get('CONTENT_LENGTH', 0))
    except (ValueError):
        request_body_size = 0
    requestBody = json.loads(environ['wsgi.input'].read(request_body_size).decode("utf-8"))

    #Image acquisition
    print("Get pucture")
    imageName = randomStr(10)
    imageData = base64.b64decode(requestBody["image"])
    imagePath = "/tmp/" + imageName
    with open(imagePath, 'wb') as f:
        f.write(imageData)

    #Content prediction
    print("Predicting ... ")
    result = {}
    predictions, probabilities = prediction.predictImage(imagePath, result_count=5)
    print(zip(predictions, probabilities))
    for eachPrediction, eachProbability in zip(predictions, probabilities):
        result[str(eachPrediction)] = str(eachProbability)

    return Response(start_response, result)

Required dependencies:

tensorflow==1.13.1
numpy==1.19.4
scipy==1.5.4
opencv-python==4.4.0.46
pillow==8.0.1
matplotlib==3.3.3
h5py==3.1.0
keras==2.4.3
imageai==2.1.5

Write the configuration file required for deployment:

ServerlessBookImageAIDemo:
  Component: fc
  Provider: alibaba
  Access: release
  Properties:
    Region: cn-beijing
    Service:
      Name: ServerlessBook
      Description: serverless book case
      Log: Auto
      Nas: Auto
    Function:
      Name: serverless_imageAI
      Description: image target detection
      CodeUri:
        Src: ./src
        Excludes:
          - src/.fun
          - src/model
      Handler: index.handler
      Environment:
        - Key: PYTHONUSERBASE
          Value: /mnt/auto/.fun/python
      MemorySize: 3072
      Runtime: python3
      Timeout: 60
      Triggers:
        - Name: ImageAI
          Type: HTTP
          Parameters:
            AuthType: ANONYMOUS
            Methods:
              - GET
              - POST
              - PUT
            Domains:
              - Domain: Auto

In the code and configuration, you can see the existence of the directory: / MNT / Auto /. This part is actually the address after NAS is mounted. You only need to write it into the code in advance. The next step will be to create NAS and configure the mount point.

Project deployment and testing

After completing the above steps, you can:

s deploy

Deploy the project. After the deployment, you can see the results:

20 lines of code: image classification and prediction can be easily done in Python under serverless architecture

After deployment, you can:

s install docker

For dependent installations:

20 lines of code: image classification and prediction can be easily done in Python under serverless architecture

After the dependency installation is completed, you can see that the. Fun directory is generated in the directory, which is the dependency file packaged through docker. These dependencies are the dependencies declared in the requirements.txt file.

After that, we passed:

s nas sync ./src/.fun

Package and upload the dependent directory to NAS, and then package and upload the model directory after success:

s nas sync ./src/model

After completion, you can:

s nas ls --all

View catalog details:

20 lines of code: image classification and prediction can be easily done in Python under serverless architecture

After completion, we can write a script for testing, which is also applicable to the test picture just now. Through the code:

import json
import urllib.request
import base64
import time

with open("picture.jpg", 'rb') as f:
    data = base64.b64encode(f.read()).decode()

url = 'http://35685264-1295939377467795.test.functioncompute.com/'

timeStart = time.time()
print(urllib.request.urlopen(urllib.request.Request(
    url=url,
    data=json.dumps({'image': data}).encode("utf-8")
)).read().decode("utf-8"))
print("Time: ", time.time() - timeStart)

You can see the results:

{"Response": {"laptop": "71.43893837928772", "notebook": "16.265614330768585", "modem": "4.899385944008827", "hard_disc": "4.007565602660179", "mouse": "1.2981869280338287"}, "ResponseId": "1d74ae7e-298a-11eb-8374-024215000701"}
Time:  29.16020894050598

It can be seen that the function calculation successfully returned the expected results, but the overall time-consuming was more than expected, nearly 30s. At this time, let’s execute the test script again:

{"Response": {"laptop": "71.43893837928772", "notebook": "16.265614330768585", "modem": "4.899385944008827", "hard_disc": "4.007565602660179", "mouse": "1.2981869280338287"}, "ResponseId": "4b8be48a-298a-11eb-ba97-024215000501"}
Time:  1.1511380672454834

It can be seen that the execution time is only 1.15 seconds, which is as much as 28 seconds higher than the last time.

Project optimization

In the last round of test, we can see the time difference between the first start and the second start of the project. In fact, this time difference is mainly due to the extremely long time wasted when the function loads the model.

Even locally, we can simply test:

# -*- coding: utf-8 -*-

import time

timeStart = time.time()

#Model loading
from imageai.Prediction import ImagePrediction

prediction = ImagePrediction()
prediction.setModelTypeAsResNet()
prediction.setModelPath("resnet50_weights_tf_dim_ordering_tf_kernels.h5")
prediction.loadModel()
print("Load Time: ", time.time() - timeStart)
timeStart = time.time()

predictions, probabilities = prediction.predictImage("./picture.jpg", result_count=5)
for eachPrediction, eachProbability in zip(predictions, probabilities):
    print(str(eachPrediction) + " : " + str(eachProbability))
print("Predict Time: ", time.time() - timeStart)

Execution results:

Load Time:  5.549695014953613
laptop : 71.43893241882324
notebook : 16.265612840652466
modem : 4.899394512176514
hard_disc : 4.007557779550552
mouse : 1.2981942854821682
Predict Time:  0.8137111663818359

It takes only 1.5 seconds to load the model, and it takes less than 1.5 seconds to load the model. In function calculation, the machine performance itself is not as high as my local performance. At this time, in order to avoid the long response time caused by each model loading, you can see that the model loading process is actually placed outside the entry method in the deployed code. One advantage of this is that each time the project is executed, there will not necessarily be a cold start, that is, some objects can be reused on the premise of some reuse, that is, there is no need to reload the model and import dependencies every time.

Therefore, in the actual project, in order to avoid frequent requests, instance repeated loading and creation of some resources, we can put some resources during initialization. This can greatly improve the overall performance of the project, cooperate with the reserved capacity provided by the manufacturer, and basically eliminate the negative impact of function cold start.

summary

In recent years, artificial intelligence and cloud computing have developed rapidly. In serverless architecture, how to run traditional artificial intelligence projects has gradually become something that many people need to know. This paper mainly introduces an interface for image classification and prediction through an existing dependency Library (imageai). With this example, several things can be clarified:

  • Serverless architecture can run AI related projects;
  • Serverless can be well compatible with tensorflow and other machine learning / deep learning tools;
  • Although the function calculation itself has space limitations, in fact, after the hard disk mounting capacity is increased, the function calculation itself will be greatly expanded.

Of course, this article can also be regarded as a brick to attract jade. I hope readers can give play to their imagination and further combine more AI projects with serverless architecture after this article.