Pok é mon AI, use dall-e to generate Magic Baby Atlas

Time:2022-3-24

Remember the article we shared last time about using dall-e to generate magic baby? This time, reddit users gave a better version.

Every year, millions of people eagerly wait for the release of a new generation of magic babies. Despite the growing collection, the goal remains the same – gotta catch ’em all!

Pok é mon AI, use dall-e to generate Magic Baby Atlas

As a fanatical Magic Baby fan and AI fan, I have tried many times to integrate the two, but failed. A few years ago, due to the emergence of generation countermeasure network (GAN), when image generation began to flourish, I tried to manually generate Pok é mon images using the most advanced model at that time. First look at the failed works.

Pok é mon AI, use dall-e to generate Magic Baby Atlas

With the progress in the field of image generation, the quality of magic babies that can be generated is also improving. A year later, the second attempt to use progan produced slightly better results. Although it captures the clear outline of the magic baby, it seriously lacks any details.

Pok é mon AI, use dall-e to generate Magic Baby Atlas

In recent years, great progress has been made in the field of generative modeling. With the emergence of transformer and the use of self supervised learning on massive data, image generation has entered a new era. These models are usually large-scale and can use a large amount of unlabeled data to learn high-quality representation. Therefore, these models can be fine tuned on a small amount of data and use these pre trained representations to produce excellent results.

It is worth mentioning the idea of mode collapse. When the model finds the most reasonable image, it focuses too much on generating specific output, resulting in the lack of diversity of images. Early Gan was plagued by this problem, which was reflected in the lack of diversity of generated Pok é mon.

Openai’s dall-e, which is based on the transformer language model, can generate high fidelity images from text using the data set of text image pairs. Dall-e uses a combination of discrete variational self encoders (DVAE) to obtain a discrete potential representation, while using a powerful autoregressive transformer based decoder to generate high-quality images. The use of DVAE helps to avoid the problem of a posteriori collapse – potential variables are ignored when they are paired with a powerful autoregressive decoder.

What if we can manage a text image pair data set that describes Pok é mon and its related images? Then it can be fine tuned with dall-e to learn the correlation between visual concepts in the text, such as the type of magic baby, so as to control the attributes of the image it generates. For example, passing the sentence “grass poison Magic Baby” to the model will generate an image of grass and poison Magic Baby (usually depicted in green and flower shape).

In the process of completing the customs clearance of the latest game of the series “Magic Baby: bright diamond”, I noticed that each magic baby has its own 3D model. We have another good new dataset to play!

With this newly discovered knowledge, I hurried to plan a data set to fine tune dall-e. This includes the text description of the image and type of magic baby, as well as some unique properties of magic baby.

Pok é mon AI, use dall-e to generate Magic Baby Atlas

With the help of some ready-made dall-e notebooks, put together some pytorch code, and put the newly created data set into the model for several hours of fine-tuning. The model can already create some new magic babies!

Generating Pok é mon is as simple as describing the prompt model in text. Here are a bunch of examples for your entertainment.

A grass type Pokémon

Pok é mon AI, use dall-e to generate Magic Baby Atlas

A fire type Pokémon

Pok é mon AI, use dall-e to generate Magic Baby Atlas

A water type Pokémon

Pok é mon AI, use dall-e to generate Magic Baby Atlas

A dragon type Pokémon

Pok é mon AI, use dall-e to generate Magic Baby Atlas

A fairy type Pokémon

Pok é mon AI, use dall-e to generate Magic Baby Atlas

A legendary water and dragon type Pokémon

Pok é mon AI, use dall-e to generate Magic Baby Atlas

Image generation has made great progress in the past few years. From generating blurred pixel blocks to generating reasonable images of magic baby according to text description is a great feat. Although the image still needs to be repaired, it can bring us endless creativity.

These creatures have no names, so I welcome you to submit the names you think are suitable for these “magic babies”!

By Liam eloie