Zhang Dongsheng, I know it’s you! How to use Gan to make a baldness maker

Time:2021-2-18

Liu Xuan works in Fujian Zhenshi information. Python Development Engineer for 2 years.
Zhang Dongsheng, I know it's you! How to use Gan to make a baldness maker
Recently, a TV play “secret corner” has caused a lot of discussion on the Internet. It is not too much to say that it is the most popular TV play in the whole network in 2020. The villain boss Zhang Dongsheng in the play is also one of the topics discussed by netizens, especially his bald features, which has become a hindrance.

Zhang Dongsheng, I know it's you! How to use Gan to make a baldness maker

After watching the show, I suddenly wanted to know what my baldness looked like, so I checked the official website of flying oars, and sure enough, it had a model library of pictures. So, how do we use paddlepaddle to make a baldness generator.

**Zhang Dongsheng, I know it's you! How to use Gan to make a baldness maker!
Introduction of generation countermeasure network

When it comes to image generation, we have to talk about Gan, which is an unsupervised learning method, learning by making two neural networks play games with each other. This method was proposed by LAN goodflow and others in 2014. The generative confrontation network is composed of a generative network and a discriminant network. The generative network samples randomly from the latent space as the input, and its output needs to imitate the real samples in the training set as much as possible.

The input of the discrimination network is the real sample or the output of the generation network. The purpose is to distinguish the output of the generation network from the real sample as much as possible. However, the generating network tries to deceive the discriminating network as much as possible. The two networks confront each other and constantly adjust their parameters. Generative countermeasure network is often used to generate false images. In addition, the method is also used to generate films, 3D object models and so on.

There are three main models for face attribute conversion in the model library of flying oar

  • Stargan multi domain attribute migration

The common Gan model needs to establish an independent model for each pair of image style domains. In contrast, stargan is a versatile player, that is, a single stargan model can realize the conversion of multiple different style domains. It allows multiple data sets of different style domains to be used for training in a network at the same time. As a result, stargan’s image quality is better than the existing model, and it can flexibly convert the input image to any desired style field.

Zhang Dongsheng, I know it's you! How to use Gan to make a baldness maker

(a) To deal with multiple domains, a cross domain model should be established for each pair of domains.

(b) Stargan uses a single generator to learn the mapping between multiple domains. The graph represents a topology that connects multiple domains.

  • Attgan uses classification loss and reconstruction loss to ensure the change of specific attributes

Attgan algorithm is based on encoder decoder structure, decodes the potential representation of a given face according to the required attributes, and realizes the editing of face attributes. Now some methods try to create a potential representation independent of attributes to edit attributes. However, too many constraints are not related to the attributes of potential representations, which limits the ability of potential representations and may lead to information loss, resulting in over smooth and distorted generation.

Instead of imposing constraints on the potential representation, attgan applies attribute classification constraints to the generated images to ensure the correct change of the required attributes, that is, “change what you want.”. At the same time, refactoring loss is introduced to ensure that only specific attributes are changed. It can also be directly applied to attribute strength control, and can be naturally extended to attribute style operations.

  • Stgan only inputs tags with changes, and introduces Gru structure to better select changed attributes

Stgan is an improvement based on attgan. In stgan, the network is generated by adding selective transfer units (stu) between encoder and decoder and introducing attribute difference vector (only input the attributes that need to be changed). In this way, the network becomes easier to train. Compared with the target attribute tag, the attribute difference tag can provide more valuable information and significantly improve the accuracy of attribute generation.

Zhang Dongsheng, I know it's you! How to use Gan to make a baldness maker

Network structure of stgan

Stgan is almost an upgraded version of attgan. Stargan of paddle does not contain baldness attribute, so we use stgan. The preview of stgan effect is as follows:

Zhang Dongsheng, I know it's you! How to use Gan to make a baldness maker

Effect picture of stgan

In the above figure, the corresponding attributes of each avatar image are: original image, bald, bands, black hair, blond hair, brown hair, bushy eyebrows, eyeglasses, male, mouth slowly open, mustache, no beard, palm skin and young.

What do we see? Bald! Yes, this is the baldness attribute we want!

AI Studio is a free cloud computing platform provided by Baidu PaddlePaddle. It has free GPU computing power, notebook’s UI style and pre installed PaddlePaddle. It’s very suitable for me to make a new alchemist.

Now let’s start the journey of baldness in AI studio

Experimental process

1. Prepare the data set

This project uses celeba data set, celeba is the abbreviation of celebfaces attribute, which means celebrity face attribute data set. It contains 202599 face images of 10177 celebrity identities. For the detailed introduction of celeba dataset, please refer to https://zhuanlan.zhihu.com/p/… .

#Decompress the dataset
unzip data/data21325/imgAlignCeleba.zip -d dataset/
cp data/data21325/*.txt -d dataset/

2. Get the model

#Get Gan model
The paddlepaddle/models warehouse contains all the Gan models that it supports.
!git clone https://gitee.com/paddlepaddle/models.git -b release/1.8
!cp -r models/PaddleCV/gan/* ./work/

3. Start training

#Training
%cd ~/dataset
!python ../work/ train.py  --model_ Net? Stgan? Specifies the stgan model network
--data_ Dir.. / dataset \ \ dataset directory
--Dataset. \ \ # relative position of dataset
--crop_size 170 \                                    
--image_size 128 \
--train_list ../dataset/attr_celeba.txt \        #Training集位置
--gan_mode wgan  \
--batch_size 32 --print_freq 1 \
--num_discriminator_time 5 \
--epoch 50 \
--dis_norm instance_norm \
--output ~/output/stgan/

4. Start reasoning

take it easy! Before “baldness”, we need to prepare the image to be input. I put it in my_ In the dataset, modify dataset / test1.txt, fill in the image, and input the characteristics according to the characteristics of the image

Zhang Dongsheng, I know it's you! How to use Gan to make a baldness maker

%cd ~
#The input parameters can be seen in infor_ bald.py At the beginning of the explanation oh, the main need to pay attention to is
# n_ Samples: its number determines the output result. The graph contains the results of several graphs. If I set it to 1, there will be only one graph's attribute transformation output.
# crop_ size、image_ It's better not to modify the size. After my test, it will affect the effect, so it's better to control the width of the picture at 178
!python ./work/infer_bald.py \
--model_net STGAN \
--init_model ./33/ \
--dataset_dir my_dataset \
--test_list dataset/test1.txt \
--use_gru True \
--output ./infer_result/stgan/ \
--n_samples 1 \
--selected_attrs "Bald,Bangs,Black_Hair,Blond_Hair,Brown_Hair,Bushy_Eyebrows,Eyeglasses,Male,Mouth_Slightly_Open,Mustache,No_Beard,Pale_Skin,Young" \
--c_dim 13 \
--crop_size 178 \
--image_size 128 \
--load_height 128 \
--load_width 128 \
--crop_height 128 \
--crop_width 128 \

5. Baldness effect display

Zhang Dongsheng, I know it's you! How to use Gan to make a baldness maker

Actor Qin Hao (played by Zhang Dongsheng)

Zhang Dongsheng, I know it's you! How to use Gan to make a baldness maker

I feel very bald

Zhang Dongsheng, I know it's you! How to use Gan to make a baldness maker

Gradual degree of baldness

Effect conclusion

  • Because stgan only input the changed attributes, the original infer will cycle each style attribute (bald, bands, etc.), I changed the cycle of switching style attributes to only input baldness attributes, and then the cycle will gradually change the degree of change, so that the result will have a gradual effect

Zhang Dongsheng, I know it's you! How to use Gan to make a baldness maker

  • Here, I find that the baldness effect of short hair in men is obviously better than that of men with long hair, and fluffy hair will affect the baldness effect, because fluffy hair covers the shape of the head, which may also be the reason for the lack of such male pictures in the training set. The first picture is the most natural and true. It should be the factor of short hair.
  • The size of the input image is close to 128×128, or 178×178, the effect will be better, probably because the size of the training set is 128×128. The official here also states:( http://https//www.paddlepaddl…_ celeba&en_ category=GANs)

    • Try to only show your face in the image to be processed. When the facial features are facing the front and showing the facial features, the effect will be better.
    • When the size of the image to be processed is close to 128 * 128 pixels, the effect will be better.

Using paddlehub

If you think the above is more complicated and the code in infer is complex, then there is a direct shortcut. Paddlehub already has stgan’s pre training model, which can be used directly.

#Install paddlehub and stgan_ Celeba pre training model
!pip install paddlehub==1.6.1 -i https://pypi.tuna.tsinghua.edu.cn/simple
!hub install stgan_celeba
import paddlehub as hub

stgan = hub.Module(name="stgan_celeba")

test_img_path = ["my_dataset/img_align_celeba/000003.jpg"]
# org_ Info is a list with only one element, such as: "bald, bands"]
# org_ Info to explain the characteristics of the input image as detailed as possible, otherwise the output effect will be affected
#Gender ("male" or "female") must be filled in. The optional values are "bald", "bands", "black"_ Hair", #"Blond_ Hair", "Brown_ Hair", "Bushy_ Eyebrows", "Eyeglasses", #"Mouth_ Slightly_ Open", "Mustache", "No_ Beard", "Pale_ Skin", "Aged"
org_info = ["Male"]
#Specifies the characteristic to change: baldness
trans_attr = ["Bald"]

# set input dict
input_dict = {"image": test_img_path, "style": trans_attr, "info": org_info}

# execute predict and print the result
results = stgan.generate(data=input_dict)
print(results)

Comparison of pre training model and my own training model:

Zhang Dongsheng, I know it's you! How to use Gan to make a baldness maker

My own

Zhang Dongsheng, I know it's you! How to use Gan to make a baldness maker

Pre training model

It can be seen that the degree of baldness of the pre training model is relatively fixed, but I set the degree of baldness smaller, which may be more realistic.

thank

Finally, thanks to the flying paddle platform, I can do some interesting experiments as a beginner.

And I’m very interested in stylegan. I hope I can support it in the future.

If there is any problem in the use process, you can join the official QQ group of flying propeller to communicate: 1108045677.

If you want to learn more about the flying oars, please refer to the following documents.

Project address of propeller generation countermeasure network:

GitHub: 

https://github.com/PaddlePadd…

Gitee:

https://gitee.com/paddlepaddl…

Official website address:

https://www.paddlepaddle.org.cn

Address of the project:

GitHub:

https://github.com/PaddlePadd…

Gitee: 

https://gitee.com/paddlepaddl…

END