TokenFlow: Consistent diffusion features for consistent video editing

https://diffusion-tokenflow.github.io/

by puttycat • 3 years ago

173 15 3 years ago

diffusion-tokenflow.github.io

KETpXDDzR• 3 years ago

"code coming soon" I really really hope this code will be ready is setup and run. My experience with ML/AI research GitHub code is pretty bad. Most of the time you have to guess the right dependencies (and versions).

dooraven• 3 years ago

yeah it's especially annoying on which CUDA version you have to guess to make it work. Sometimes projects don't even produce conda environments properly.

londons_explore2• 3 years ago

I would prefer incomplete or barely working code over no code.

GitHub is the perfect place to post incomplete code, because someone else can contribute fixes.

If you're a scientist, please don't hold back publishing code just because someone else demands a certain code quality.

yreg• 3 years ago

95% of problems would be solved by locking the entire dependency tree.

throwaway14356• 3 years ago

long long ago i had some fun stitching frames into a panorama from some really old footage (by hand). The most hilarious trick was to upscale the footage a lot so that frames could be visually aligned, made transparent and improve the resolution. The grainy snow turned into a sharp picture with new previously invisible details

yieldcrv3• 3 years ago

they messed up AWS access to the videos, as of time of writing

they should upload to ipfs, pin with filecoin, and let a CDN like cloudfront’s serve the video

londons_explore2• 3 years ago

Or... Just upload to YouTube and embed it.

And YouTube will deal with transcoding to different resolutions for different devices, auto select what quality to send depending on bandwidth, ensure it works on all popular platforms, and all the other bits of video hosting that these researchers didn't want to do.

yieldcrv3• 3 years ago

I was assuming they had a reason for not doing the simple thing and offered something similar

jmacc93• 3 years ago

This webpage caused my laptop (4 GB) to OOM. Really cool though!

als0• 3 years ago

My browser tries to load all the videos at once...can't watch any of them yet.

totetsu• 3 years ago

This instagram account makes some really neat stuff leaning into temporal inconsistencies

https://www.instagram.com/never_ever_never_land/

zhaosk• 3 years ago

ohhhh, why not try CDN to serve the video

skerit• 3 years ago

The more I see things like this, the more I wonder why there is no good video upscaler yet. Something that can really upscale old DVD or even VHS-quality video.

Topaz Video Enhance AI has barely advanced in the past few years, old versions sometimes even produce better results than new versions. And even then, some of the enhancements can be achieved with some carefully chosen non-AI filters.

danielbln• 3 years ago

We're getting there. Temporally stable video generation is not trivial and is only now slowly solved. Upscaling of single images in a generative fashion (as in, actually adding in details) works really well, you can create images with an incredible amount of detail. Doing the same for video is around the corner.

yieldcrv3• 3 years ago

The videos are working now! Wow! Getting closer to perfection!

In some styles this is good enough!

TokenFlow: Consistent diffusion features for consistent video editing

Related Stories