I recently got the question from my uncle whether I can upscale one of his really old videos. The source was a short 10 seconds video with some low quality audio in 320×240 pixel resolution. Likely taken by one of the first video capable digital cameras or a phone many years ago.

I accepted the challenge as I had seen some AI tools like DiffusionBee being able to upscale images with decent quality.

I haven’t found a good free tool to upscale a video directly yet. There are shade free tools out there, but I don’t trust them.

What I ended up doing is exporting each frame of the original video to an image, scale up the images with an open AI model and then stitch them back together to a video.

1.) export each frame of the video to a JPEG file, export sound into a single file

ffmpeg -i input.mp4 ./LOW/frame_%04d.jpg

ffmpeg -i input_video.mpeg -vn ./output_audio.mp3

Directory Structure:

.
├── HIGH
│   ├── upscayl_jpg_realesrgan-x4plus_4x
│   │   ├── frame_0001.jpg
│   │   ├── frame_0002.jpg
│   │   ├── frame_0003.jpg
...
│   │   ├── frame_0254.jpg
│   │   └── frame_0255.jpg
│   ├── upscayl_jpg_remacri_3x
│   ├── upscayl_jpg_ultramix_balanced_3x
│   └── upscayl_jpg_ultrasharp_2x
└── LOW
│   ├── frame_0001.jpg
│   ├── frame_0002.jpg
...
│   ├── frame_0254.jpg
│   └── frame_0255.jpg

2.) Upscale images using AI tool Upscaly

https://upscayl.org/

brew install --cask upscayl

3.) combine new images into a movie

cd ./HIGH/upscayl_jpg_realesrgan-x4plus_4x

ffmpeg -framerate 15 -f image2 -pattern_type glob -i "frame_?.jpg" -i ../../output_audio.mp3 -c:v libx264 -crf 1 -vf scale=2048:2048 -pix_fmt yuv420p -vb 100M ../output_${PWD##/}.mp4