VToonify is an AI software that cartoonizes faces in videos. It is built on StyleGAN. You can find the original paper here. This tutorial gives you a step-by-step guide on how to turn a video into a cartoon for free using VToonify in GitHub. (Note: VToonify works best with face-focus videos.)
1. First download the code from VToonify GitHub repository. Open a dos prompt. In the directory where you want to install, type the command:
>git clone https://github.com/williamyang1991/VToonify.git
2. Read README.md in the “VToonify\checkpoint” directory. It tells you the folder structure of the checkpoints.
3. Now download the checkpoints. Go to hugging face to download “encoder.pt”, “faceparsing.pth” and “shape_predictor_68_face_landmarks.dat.” Go to Google drive to download “directions.npy.” Put them under the “checkpoint” root directory.
4. There are a few pre-trained models for different styles, such as cartoons, comics, illustrations, Pixar, etc. You can see the samples of each style in huggingface space. I suggest you try one of them first. “vtoonify_d_cartoon” has the best results. Go to vtoonify_d_cartoon at hugging face, download four files there, and put them under your “checkpoint\vtoonify_d_cartoon” directory.
5. Now set up a conda environment to run VToonify with this instruction. If you feel lost when you set up, please refer to the post Install PyTorch GPU on Windows – A complete guide before running the conda command to create a new virtual environment.
6. If you are going to run with Nvidia GPU on Windows, you need to install VS2022 buildtools. Set the path of cl.exe in the environment variables. The instructions and issues are explained here.
7. You can also run the code using the CPU. The speed is acceptable. To run on CPU, you change 3 files: “VToonify\util.py”, “VToonify\model\simple_augment.py”, “VToonify/model/stylegan/model.py.” In these files, change “model.stylegan.op” to “model.stylegan.op_cpu.” See the detailed instructions here.
8. Prepare a short video mp4 or gif file with a clear face movement. Put it in the “data” directory. You can also use the existing input file in “data” as well, for example, “651.mp4.”
9. Open an Anaconda Prompt. Run command (I name my env to be “stylegan3_env”):
>conda activate stylegan3_env
10. Still in the Anaconda prompt, go to the directory “VToonify” and run the command:
>python style_transfer.py --scale_image --content ./data/651.mp --video
11. When it finishes, the new video is saved in the “output” directory. If you cannot play the output mp4 file due to the encoding differences, import the file to After Effects or other video editing tools to render it again as a video.
12. (Optional) Now you can download other pre-trained models and check out their results.
