Alibaba cloud video cloud creates a civilian “virtual human” with inclusive Technology


Recently, ayayi, the first super realistic digital person in China, announced that he would join Alibaba and become the digital manager of tmall super brand day. They will jointly open the marketing world of metauniverse. In the future, she will unlock multiple identities with tmall, such as NFT artist, digital curator, tide brand manager, top stream digital person, etc.

Alibaba cloud video cloud creates a civilian

When it comes to virtual digital people, we first think of Luo Tianyi, who was born in 2012. Her appearance fills the gap in the domestic virtual idol market. Although she only had sound and vertical painting at first, over the past nine years, fans have been writing songs and music, drawing illustrations and making MMD for her. These colleagues have filled the initial blank little by little. Compared with real person idols, virtual idols growing up relying on UGC content are more like a pure nurturing star making. At the beginning of 2021, the dimensional virtual idols represented by Luo Tianyi began to shine in the Spring Festival Gala, which means that the national platform respects and recognizes the dimensional culture, and it is also a beneficial interaction between virtual and reality.

Alibaba cloud video cloud creates a civilian
Luo Tianyi appeared in the Spring Festival Gala program children’s song and dance “listen to me”

According to iimedia research, the scale of China’s virtual idol core industry in 2020 was 3.46 billion yuan, an increase of 70.3% year-on-year, and it is expected to reach 6.22 billion yuan in 2021; On the other hand, with the continuous exploration of commercial value, more and more industries are connected with virtual idols. The industrial scale driven by virtual idols will be 64.56 billion yuan in 2020 and 107.49 billion yuan in 2021.
In foreign short video platforms, virtual digital people are also popular. A virtual idol named lil Miquela is a 19-year-old mixed race girl with lovely freckles. She has long settled in Los Angeles and has virtual models and musicians of Hispanic, Brazilian and American descent. According to the report of CCTV on January 30, 2021, lil Miquela’s revenue in 2019 is about 76 million yuan.

What is virtual digital human?

The term “virtual digital man” originated from the “visible human project” (yhp) launched by the National Medical Library of the United States in 1989. In 2001, the concept of “digital virtual human body” was put forward at the 174th Symposium of Xiangshan Science Conference with the theme of “scientific and technological problems of China’s digital virtual human body”. These “virtual digital people” mainly refer to the visualization of human body structure, which displays the size, shape, position of human anatomical structure and the mutual spatial relationship between organs in three-dimensional form, that is, using human body information to realize the digitization of human anatomical structure. It is mainly used in human anatomy teaching, clinical diagnosis and treatment in the medical field.

Different from the digital human body in the above medical field, the virtual digital human (hereinafter referred to as “digital human”) discussed around the meta universe on the network refers to the virtual human with digital shape. Unlike robots with entities, virtual digital humans rely on display devices. Virtual digital human should have the following three characteristics: first, the appearance of the owner, with specific characteristics such as appearance, gender and personality; Second, the owner’s behavior has the ability to express in language, facial expressions and body movements; Third, the owner’s thought has the ability to identify the external environment and interact with people.

In recent years, thanks to the breakthrough of deep learning algorithm, the production process of digital people has become simpler. Virtual digital people have begun to enter the public’s vision and are applied in all walks of life. For example, there have been entertainment digital people (such as virtual anchor and virtual Idol), educational digital people (such as virtual teacher), assistant digital people (such as virtual customer service, virtual tour guide and intelligent assistant) Film and television digital people (such as double actors or virtual actors), etc. Virtual digital people with different shapes and functions enable the fields of film and television, media, games, finance, culture and tourism to provide customized services for users according to their needs.

Alibaba cloud video cloud creates a civilian
Virtual digital human industry view source: 2020 white paper on virtual digital human development

In terms of implementation technology, the virtual digital human system is generally composed of five modules: character image, voice generation, animation generation, audio and video synthesis display, interaction and so on. Among them, the most basic is the character image technology. According to the dimension of character graphics resources, it can be divided into 2D and 3D. From the appearance, it can be divided into cartoon, anthropomorphic, realistic, super realistic and other styles.

Alibaba cloud video cloud creates a civilian
Source: 2020 white paper on virtual digital human development

Alibaba cloud cartoon smart painting: creating virtual idols around you

Because the character image construction of virtual human has a certain technical threshold, it seems that for ordinary people, virtual human still stays in the idol stage and can only “look from a distance”. However, through optimized algorithm technology, the team from Alibaba cloud video cloud gives ordinary people the opportunity to create their own cartoon images and create “civilian idols” in the virtual world. In the developer exhibition area of yunqi conference in 2021, Alibaba cloud showed the latest technology – cartoon smart drawing project, which attracted nearly 2000 experimenters and became the hot money of the conference. This technology adopts the face stylization technology scheme of implicit variable mapping. Through the machine learning algorithm, the original facial features of the photographer and the target training style can be deeply fused to reconstruct the shape of the face. It not only conforms to the user’s facial features, but also has the style of cartoon. It can also track the user’s facial posture and rich expressions to generate real-time animation.

Alibaba cloud video cloud creates a civilian
All these technologies are completed by AI. For users, they can automatically generate virtual humans that meet the characteristics of users’ faces by uploading a self photo.
Alibaba cloud video cloud creates a civilian

Alibaba cloud cartoon smart drawing can automatically generate a virtual image with personal characteristics (i.e. the effect after stylization) by inputting a portrait image, exploring its salient features (such as eye size, nose shape, etc.). The so-called face stylization is to convert the real face avatar into a specific style avatar. The most common include cartoon style, animation style and oil painting style, as shown in the figure below:

Alibaba cloud video cloud creates a civilian
Alibaba cloud video cloud cartoon intelligent rendering

Thanks to the rapid development of AI technology, virtual human is no longer out of reach. In the foreseeable future, virtual human created by cartoon intelligent drawing will appear more and more frequently in all walks of life as the digital avatar of virtual anchor, virtual teacher, virtual customer service and virtual tour guide.

Reference: general group of China artificial intelligence industry development alliance and digital human working committee of Zhongguancun digital intelligence industry alliance, white paper on virtual digital human development in 2020

“Video cloud technology”, the official account of your most noteworthy audio and video technology, is pushing practical technical articles from Ali cloud every week. The official account is back to technology, and it can join Ali cloud video cloud products technology exchange group.