The way of image recording gives us an opportunity to intuitively feel the history. Pictures and videos carry the memory of generations. The use of coloring and restoration technology to repair old photos has become a new way for us to recall our relatives and history.
Recently, a black-and-white image restoration tool named deoldify on GitHub has attracted a lot of attention. At present, the number of stars has exceeded 10000. It is worth noting that this project is open source!
Each of the three models has its own advantages
Deoldify can color and restore old images and films. Three models can be selected in deoldify, each of which has key advantages and disadvantages, so it has different use cases.
Artistic model:The model achieves the highest quality results in interesting details and brightness. However, the most obvious drawback is that it takes some effort to get the best results, and you have to adjust the rendering resolution or render_ Factor. The model does not perform well in some key common scenes, including natural scenes and portraits. It focuses on the layer depth of the decoder.
Stability model:The model can get the best effect in horizontal and vertical conditions, and usually has less weird colors than art colors. The model focuses on the layer width on the decoder side.
Video model:This model is optimized for smooth, consistent and flicker free video. This must be the least colorful of the three models, but to be honest, it’s not far from “stable.”. The model is the same as “stable” in architecture, but different in training.
Although the old photo restoration technology gives us a chance to see the real images of the past ten years or even hundreds of years, there may still be a problem, that is, whether the color restoration of these photos and videos is accurate?
Take the following picture as an example. In the restored photo, the color of the bridge is white, but after investigation, the bridge is actually red. In other words, historical accuracy is still a huge challenge for the old photo restoration technology.
What is nogan? Why use it to make the image color more stable?
What is nogan?
This is a new Gan training developed by the creator of deoldify, which is used to solve some key problems in the previous deoldify model. Most of the training time is spent on pre training the generator and annotator respectively through more direct, faster and more reliable conventional methods. This method can eliminate faults and artifacts.
Original deoldify model:
Nogan based deoldify model:
In image inpainting technology, it is very difficult to restore video image stably. Deoldify adopts nogan training combined with Gan training, which can not only provide stable color image restoration, but also eliminate flicker in video.
Video is rendered using isolated image generation without any additional time modeling, using 1% to 3% of Imagenet data at a time. Then, like still image coloring, each frame is “deoldify” processed before video reconstruction.
In addition to improving video stability, there is another interesting thing to mention. Although different models and models with different training structures all come to the same solution more or less. This is true even when coloring arbitrary and unknowable things, such as clothes, car colors, and even special effects colors.
In response, the creators of deoldify speculate that the models are learning some interesting rules to color based on the subtle cues that exist in black-and-white images. Even in a moving scene, the results of these renderings are very consistent.
Statement on open source support
Open source has brought many benefits to the world, and the creation of deoldify also benefits from open source.
“Our position is that the research code and documentation we provide is beneficial to the world,” the founder of deoldify said in a statement. What we offer is new performance of colorization, Wan and video. We hope it will be helpful for developers and researchers to learn and adopt. “
They don’t plan to provide a free “product” or “application” that can be used at any time, and they don’t plan to provide such a service in the future. “Deoldify will continue to be a Linux based project, without windows support, coding in Python, and requiring people to have some additional technical background to use it,” they said
Now, some people have developed their own applications through deoldify, some paid, some free. In this regard, the position of the founder of deoldify is that as long as you have the appropriate background and resources, the project will provide you with enough entry resources.
This project is built around the fast. AI library, and now it can be installed simply with anaconda.
The specific steps are as follows:
Open the command line and navigate to the root folder you want to install, then type the following command:
Then start running with the following command:
Reference link: https://hackernoon.com/deoldify-can-colorize-your-black-and-white-photos-with-full-photorealistic-renders-5k2i33c3
GitHub address: https://github.com/jantic/DeOldify#a -statement-on-open-source-support