Avatar cartoon wind processing applet based on serverless architecture

Time:2022-5-26

Author: Liu Yu (flower name: Jiang Yu)

preface

I’ve always wanted to have a cartoon version of the avatar. However, my hands are too stupid and I can’t “pinch it out” with a lot of software. So I was thinking, can I implement such a function based on AI and deploy it to the serverless architecture for more people to try?

Backend project

The back-end project adopts the V2 version of animegan, the famous animation style conversion filter Library in the industry, and the effect is roughly as follows:

在这里插入图片描述

The specific information about this model will not be introduced and explained in detail here. By combining with Python web framework, AI model is exposed through interface:

from PIL import Image
import io
import torch
import base64
import bottle
import random
import json
cacheDir = '/tmp/'
modelDir = './model/bryandlee_animegan2-pytorch_main'
getModel = lambda modelName: torch.hub.load(modelDir, "generator", pretrained=modelName, source='local')
models = {
    'celeba_distill': getModel('celeba_distill'),
    'face_paint_512_v1': getModel('face_paint_512_v1'),
    'face_paint_512_v2': getModel('face_paint_512_v2'),
    'paprika': getModel('paprika')
}
randomStr = lambda num=5: "".join(random.sample('abcdefghijklmnopqrstuvwxyz', num))
face2paint = torch.hub.load(modelDir, "face2paint", size=512, source='local')
@bottle.route('/images/comic_style', method='POST')
def getComicStyle():
    result = {}
    try:
        postData = json.loads(bottle.request.body.read().decode("utf-8"))
        style = postData.get("style", 'celeba_distill')
        image = postData.get("image")
        localName = randomStr(10)
        #Image acquisition
        imagePath = cacheDir + localName
        with open(imagePath, 'wb') as f:
            f.write(base64.b64decode(image))
        #Content prediction
        model = models[style]
        imgAttr = Image.open(imagePath).convert("RGB")
        outAttr = face2paint(model, imgAttr)
        img_buffer = io.BytesIO()
        outAttr.save(img_buffer, format='JPEG')
        byte_data = img_buffer.getvalue()
        img_buffer.close()
        result["photo"] = 'data:image/jpg;base64, %s' % base64.b64encode(byte_data).decode()
    except Exception as e:
        print("ERROR: ", e)
        result["error"] = True
    return result
app = bottle.default_app()
if __name__ == "__main__":
    bottle.run(host='localhost', port=8099)

The whole code is partially improved based on serverless architecture:

  1. When the instance is initialized, the model is loaded, which may reduce the impact of frequent cold start;
  2. In the function mode, only the / tmp directory is writable, so the pictures will be cached in the / tmp directory;
  3. Although the function calculation is “stateless”, in fact, it is also reused. All data are randomly named when stored in TMP;
  4. Although some cloud vendors support binary file upload, most serverless architectures are not friendly to binary upload, so Base64 upload is still adopted here;

The above code is more AI related. In addition, it also needs an interface to obtain model list, model path and other related information:

import bottle
@bottle.route('/system/styles', method='GET')
def styles():
    return {
      "Ai animation style":{
        'color': 'red',
        'detailList': {
          "Style 1":{
            'uri': "images/comic_style",
            'name': 'celeba_distill',
            'color': 'orange',
            'preview': 'https://serverless-article-picture.oss-cn-hangzhou.aliyuncs.com/1647773808708_20220320105649389392.png'
          },
          "Style 2":{
            'uri': "images/comic_style",
            'name': 'face_paint_512_v1',
            'color': 'blue',
            'preview': 'https://serverless-article-picture.oss-cn-hangzhou.aliyuncs.com/1647773875279_20220320105756071508.png'
          },
          "Style 3":{
            'uri': "images/comic_style",
            'name': 'face_paint_512_v2',
            'color': 'pink',
            'preview': 'https://serverless-article-picture.oss-cn-hangzhou.aliyuncs.com/1647773926924_20220320105847286510.png'
          },
          "Style 4":{
            'uri': "images/comic_style",
            'name': 'paprika',
            'color': 'cyan',
            'preview': 'https://serverless-article-picture.oss-cn-hangzhou.aliyuncs.com/1647773976277_20220320105936594662.png'
          },
        }
      },
    }
app = bottle.default_app()
if __name__ == "__main__":
    bottle.run(host='localhost', port=8099)

As you can see, my approach at this time is to add a function as a new interface, so why not add such an interface in the project just now? But to maintain one more function?

  1. The loading speed of AI model is slow. If the interface for obtaining AI processing list is integrated, the performance of the interface will be affected;
    2. The AI model needs more memory to be configured, while the interface to obtain the AI processing list needs very little memory, and the memory will have a certain relationship with the billing, so separation will help to reduce the cost;

The second interface (the interface for obtaining AI processing list) is relatively simple and no problem, but there are some headache points for the interface of the first AI model:

  1. The dependencies required by the model may involve some binary compilation process, so it cannot be used directly across platforms;
  2. The model file is relatively large (more than 800m for pytoch alone), and the upload code of function calculation is only 100m at most, so this project cannot be uploaded directly;

Therefore, we need to use the serverless devs project for processing:

reference resourceshttps://www.serverless-devs.com/fc/yaml/readme

Complete the preparation of s.yaml:

edition: 1.0.0
name: start-ai
access: "default"
Vars: # global variable
  region: cn-hangzhou
  service:
    name: ai
    Nasconfig: # NAS configuration. After configuration, function can access the specified NAS
      Userid: 10003 # userid, the default is 10003
      Groupid: 10003 # groupid, the default is 10003
      Mountpoints: # directory configuration
        - serverAddr: 0fe764bf9d-kci94. cn-hangzhou. nas. aliyuncs. COM # NAS server address
          nasDir: /python3
          fcDir: /mnt/python3
    vpcConfig:
      vpcId: vpc-bp1rmyncqxoagiyqnbcxk
      securityGroupId: sg-bp1dpxwusntfryekord6
      vswitchIds:
        - vsw-bp1wqgi5lptlmk8nk5yi0
services:
  image:
    component:  fc
    Props: # component property value
      region: ${vars.region}
      service: ${vars.service}
      function:
        name: image_server
        Description: image processing service
        runtime: python3
        codeUri: ./
        ossBucket: temp-code-cn-hangzhou
        handler: index.app
        memorySize: 3072
        timeout: 300
        environmentVariables:
          PYTHONUSERBASE: /mnt/python3/python
      triggers:
        - name: httpTrigger
          type: http
          config:
            authType: anonymous
            methods:
              - GET
              - POST
              - PUT
      customDomains:
        - domainName: avatar.aialbum.net
          protocol: HTTP
          routeConfigs:
            - path: /*

Then proceed to:

1. Dependent installation: s build — use docker
2. Project deployment: s deploy
3. Create directory in NAS and upload dependency:

s nas command mkdir /mnt/python3/python
S NAS upload - R local dependency path / MNT / python3 / Python

After completion, the project can be tested through the interface.

In addition, wechat applet needs HTTPS background interface, so HTTPS related certificate information needs to be configured here, which is not expanded here.

Applet project

The applet project still adopts colorui, and the whole project has only one page:
在这里插入图片描述

Page related layout:

Step 1: select a picture
    
  
  
    
      Upload pictures locally
      Get current Avatar
    
  
  
    
      
    
    *Click the picture to preview and long press the picture to edit
  
  
    
      Step 2: select a picture processing scheme
    
  
  
    
      
        
          {{style}}
        
      
    
  
  
     
      
      {{substyle}}
    
    *Long press the style circle to preview the template effect
  
  
    {{userchosephoho? (getphotostatus? 'AI will take a long time': 'generate picture'): 'please select picture first'}}
  
  
    
      Generate results
    
  
  
    
      The service is temporarily unavailable. Please try again later
      Or contact developer wechat: zhihuiyushaiqi
    
    
      
        
      
      *Click the picture to preview, and long press the picture to save
    
  
  
    Proudly built with serverless devs
    Powered By Anycodes {{""}}
  
  
  
    
      Author's words
      
        
      
    
    
      Hello, I'm Liu Yu. Thank you for your attention and use of this small program. This small program is a avatar generation tool I made in my spare time. It's based on "artificial mental retardation" technology. Anyway, it's awkward now, but I'll try to make this small program "intelligent". If you have any good suggestions, you are welcome to contact my email or wechat. In addition, it is worth mentioning that this project is based on Alibaba cloud serverless architecture and built through serverless devs developer tool.
    
  


  
    
      
        
          
        
      
    
    
      close preview
    
  


The page logic is also relatively simple:
// index.js
//Get application instance
const app = getApp()
Page({
  data: {
    styleList: {},
    Currentstyle: "anime style",
    Currentsubstyle: "V1 model",
    userChosePhoho: undefined,
    resultPhoto: undefined,
    previewStyle: undefined,
    getPhotoStatus: false
  },
  //Event handler
  bindViewTap() {
    wx.navigateTo({
      url: '../logs/logs'
    })
  },
  onLoad() {
    const that = this
    wx.showLoading({
      Title: 'loading',
    })
    app.doRequest(`system/styles`, {}, option = {
      method: "GET"
    }).then(function (result) {
      wx.hideLoading()
      that.setData({
        styleList: result,
        currentStyle: Object.keys(result)[0],
        currentSubStyle: Object.keys(result[Object.keys(result)[0]].detailList)[0],
      })
    })
  },
  changeStyle(attr) {
    this.setData({
      "currentStyle": attr.currentTarget.dataset.style || this.data.currentStyle,
      "currentSubStyle": attr.currentTarget.dataset.substyle || Object.keys(this.data.styleList[attr.currentTarget.dataset.style].detailList)[0]
    })
  },
  chosePhoto() {
    const that = this
    wx.chooseImage({
      count: 1,
      sizeType: ['compressed'],
      sourceType: ['album', 'camera'],
      complete(res) {
        that.setData({
          userChosePhoho: res.tempFilePaths[0],
          resultPhoto: undefined
        })
      }
    })
  },
  headimgHD(imageUrl) {
    imageUrl = imageUrl. split('/'); // Cut the path of the avatar into an array
    //Convert the size value of 46 | 64 | 96 | 132 to 0
    if (imageUrl[imageUrl.length - 1] && (imageUrl[imageUrl.length - 1] == 46 || imageUrl[imageUrl.length - 1] == 64 || imageUrl[imageUrl.length - 1] == 96 || imageUrl[imageUrl.length - 1] == 132)) {
      imageUrl[imageUrl.length - 1] = 0;
    }
    imageUrl = imageUrl. join('/'); // Rewiring to string
    return imageUrl;
  },
  getUserAvatar() {
    const that = this
    wx.getUserProfile({
      Desc: "get your avatar",
      success(res) {
        const newAvatar = that.headimgHD(res.userInfo.avatarUrl)
        wx.getImageInfo({
          src: newAvatar,
          success(res) {
            that.setData({
                    userChosePhoho: res.path,
                    resultPhoto: undefined
                  })
          }
        })
      }
    })
  },
  previewImage(e) {
    wx.previewImage({
      urls: [e.currentTarget.dataset.image]
    })
  },
  editImage() {
    const that = this
    wx.editImage({
      src: this.data.userChosePhoho,
      success(res) {
        that.setData({
          userChosePhoho: res.tempFilePath
        })
      }
    })
  },
  getNewPhoto() {
    const that = this
    wx.showLoading({
      Title: 'picture generation in progress',
    })
    this.setData({
      getPhotoStatus: true
    })
    app.doRequest(this.data.styleList[this.data.currentStyle].detailList[this.data.currentSubStyle].uri, {
      style: this.data.styleList[this.data.currentStyle].detailList[this.data.currentSubStyle].name,
      image: wx.getFileSystemManager().readFileSync(this.data.userChosePhoho, "base64")
    }, option = {
      method: "POST"
    }).then(function (result) {
      wx.hideLoading()
      that.setData({
        resultPhoto: result.error ? "error" : result.photo,
        getPhotoStatus: false
      })
    })
  },
  saveImage() {
    wx.saveImageToPhotosAlbum({
      filePath: this.data.resultPhoto,
      success(res) {
        wx.showToast({
          Title: "saved successfully"
        })
      },
      fail(res) {
        wx.showToast({
          Title: "exception, try again later"
        })
      }
    })
  },
  onShareAppMessage: function () {
    return {
      Title: "rational personality Avatar",
    }
  },
  onShareTimeline() {
    return {
      Title: "rational personality Avatar",
    }
  },
  showModal(e) {
    if(e.currentTarget.dataset.target=="Image"){
      const previewSubStyle = e.currentTarget.dataset.substyle
      const previewSubStyleUrl = this.data.styleList[this.data.currentStyle].detailList[previewSubStyle].preview
      if(previewSubStyleUrl){
        this.setData({
          previewStyle: previewSubStyleUrl
        })
      }else{
        wx.showToast({
          Title: "no template preview",
          icon: "error"
        })
        return 
      }
    }
    this.setData({
      modalName: e.currentTarget.dataset.target
    })
  },
  hideModal(e) {
    this.setData({
      modalName: null
    })
  },
  copyData(e) {
    wx.setClipboardData({
      data: e.currentTarget.dataset.data,
      success(res) {
        wx.showModal({
          Title: 'copy completed',
          Content: ` has copied ${e.currenttarget. Dataset. Data} to the clipboard ',
        })
      }
    })
  },
})

Because the project will request background interfaces for many times, I will request methods for additional abstraction:

//Unified request interface
  doRequest: async function (uri, data, option) {
    const that = this
    return new Promise((resolve, reject) => {
      wx.request({
        url: that.url + uri,
        data: data,
        header: {
          "Content-Type": 'application/json',
        },
        method: option && option.method ? option.method : "POST",
        success: function (res) {
          resolve(res.data)
        },
        fail: function (res) {
          reject(null)
        }
      })
    })
  }

After that, configure the background interface and publish it for approval.
Release the latest information of cloud native technology, collect the most complete content of cloud native technology, regularly hold live streaming and live broadcasting of cloud native life, and release the best practices of Alibaba products and users. Explore cloud native technology with you and share the cloud native content you need.

Follow [Alibaba cloud native] official account to get more real-time information about cloud native!