Blockbuster both visual sense! An open source library for quickly generating Hollywood special effects videos

Time:2021-7-28

[introduction]: skyar is a vision based method for video sky replacement and coordination. The real sky background in a video is replaced by a movie scene similar to that in a science fiction film through AI. The effect is very shocking.

brief introduction

Skyar is a vision based video sky replacement and coordination method, which can automatically generate realistic and vivid sky background in controllable style video. The author is Zhengxiao   Zou。 This method is completely based on vision, can process non-static images, has no requirements for shooting equipment and no user interaction, and can process online or offline video well. The processing process of this algorithm mainly includes sky rendering, motion estimation and image mixing.

Project address

https://github.com/jiupinjia/…

Download and install

1. Install the following dependent Libraries

Blockbuster both visual sense! An open source library for quickly generating Hollywood special effects videos
2. Download the project source code

git clone https://github.com/jiupinjia/SkyAR.git 
cd SkyAR

3. Download the pre training model and decompress it (required in China)

https://drive.google.com/file/d/1COMROzwR4R_7mym6DL9LXhHQlJmJaV0J/view?usp=sharing
unzip checkpoints_G_coord_resnet50.zip

Simple use

We directly use the video and sky templates provided by the project to experience:

1. Use the spaceship in the film “zone 9” as the sky background template

python skymagic.py --path ./config/config-canyon-district9ship.json

The effects are as follows:

Blockbuster both visual sense! An open source library for quickly generating Hollywood special effects videos

2. Use the super moon as the sky background template

python skymagic.py --path ./config/config-annarbor-supermoon.json

The effects are as follows:

Blockbuster both visual sense! An open source library for quickly generating Hollywood special effects videos

Modify the configuration to use custom video and sky templates

In the config directory, there are some configuration files in JSON format. We can change the video and sky templates according to our own situation. Take the modification of config-canyon-galaxy.json as an example:

{
  "net_G": "coord_resnet50",
  "ckptdir": "./checkpoints_G_coord_resnet50",
  "input_mode": "video",
  "datadir": "./test_videos/annarbor.mp4",
  "skybox": "floatingcastle.jpg",
  "in_size_w": 384,
  "in_size_h": 384,
  "out_size_w": 845,
  "out_size_h": 480,
  "skybox_center_crop": 0.5,
  "auto_light_matching": false,
  "relighting_factor": 0.8,
  "recoloring_factor": 0.5,
  "halo_effect": true,
  "output_dir": "./eval_output",
  "save_jpgs": false
}

Ckptdir is the directory after decompressing the pre training model file, dataDir is the directory of the input video, skybox is the sky template used, and the generated effect video is saved to output\_ Dir directory.

After modifying the configuration file, run it directly with the following command:

python skymagic.py --path .\config\config-canyon-galaxy.json

Use the floating castle as the sky background. The effect is as follows:

Blockbuster both visual sense! An open source library for quickly generating Hollywood special effects videos

There are relevant official videos available on the official website. Interested students can try.

Open source outpostShare popular, interesting and practical open source projects on a daily basis. Participate in maintaining the open source technology resource library of 100000 + star, including python, Java, C / C + +, go, JS, CSS, node.js, PHP,. Net, etc.

Recommended Today

VBS obtains the operating system and its version number

VBS obtains the operating system and its version number ? 1 2 3 4 5 6 7 8 9 10 11 12 ‘************************************** ‘*by r05e ‘* operating system and its version number ‘************************************** strComputer = “.” Set objWMIService = GetObject(“winmgmts:” _  & “{impersonationLevel=impersonate}!\\” & strComputer & “\root\cimv2”) Set colOperatingSystems = objWMIService.ExecQuery _  (“Select * from […]