Generative Image Dynamics

https://generative-dynamics.github.io/

by hughes • 3 years ago

346 29 3 years ago

generative-dynamics.github.io

Hard_Space• 3 years ago

This suffers from the same low-vector movement requirements as EbSynth.

GaggiX• 3 years ago

I think the achievement here is mostly about generating the image dynamics, so for example there is a cat in an image, the model understand that cats need to breathe so the dynamics show the lungs contracting, then the paper covers how to traslate the image dynamics and the image itself into a seamless video. I could be wrong tho

soultrees• 3 years ago

This is super cool. Cinemagraphs have always been a bit of a passion of mine, and I try to bring that feeling of subtle-stillness in a lot of the work I do, whether it’s marketing or shooting, so i can see this becoming a regular tool.

The trick to a 10/10 cinemagraph is the more subtle, the bigger the impact. You almost want the viewer to think it’s a still photo before their brain clicks in thinks “wait, something isn’t normal here, this isn’t a photo, it’s a video”

albert_e• 3 years ago

any good examples you can share please?

nico2• 3 years ago

Not op, and not sure if these qualify as cinemagraphs, but they kinda fit the “subtle image movement” description

https://twitter.com/nicobrenner/status/1685754265393041408?s...

https://twitter.com/nicobrenner/status/1686074457159274496?s...

https://twitter.com/nicobrenner/status/1685875401351217152?s...

https://twitter.com/nicobrenner/status/1686060465783443457?s...

https://twitter.com/nicobrenner/status/1685170395086045185?s...

nextaccountic• 3 years ago

there's a subreddit about that

check out www.reddit.com/r/cinemagraphs/top

CSSer2• 3 years ago

They used webGL for the demo. Nice!

sva_2• 3 years ago

This would be crazy in a video game. Walking through a bush and dragging the plant with you

CSSer2• 3 years ago

Wow, that's a neat idea! That could potentially be pretty cool. It'd almost be like a form of photogrammetry but for physics. Kinegrammetry, maybe? I wonder what the storage efficiency and performance would look like. Perhaps something like ths could be adapted into a framework for object modeling.

behnamoh• 3 years ago

I'm still waiting for video games to adopt stable diffusion, GPT, and other GenAI models. the tech is there, but I guess the inertia in the industry doesn't allow us to have nice things yet.

pelorat• 3 years ago

What do you mean, lots of games have physics that interreact with flora?

dragonwriter• 3 years ago

This would be crazy in a video game. Walking through a bush and dragging the plant with you.

But game physics can handle stuff like that, there's no need for GenAI.

divyajg• 3 years ago

I wonder why in the first picture (red rose) the flower in the bg also moves, but we don't see the same affect in the third picture (tree). I also find it impressive that the amount of motion differs in the first and the second picture, could it be because the density around the pointer is considered?

The slo-mo ones are super relaxing to watch!

actionfromafar• 3 years ago

I don't know why but I reacted with slight fear to the rose ones.

juunpp• 3 years ago

The tree has severe distortion when dragged from the edge. Still an interesting idea.

Timon3• 3 years ago

You'd probably have to combine this with segmenting and generative infill for the background layers, but luckily there's been a lot of progress there!

waffletower• 3 years ago

Nice to see Google researchers continuing to publish open papers with bonus demos. Won't beat a dead horse about Google failing to productize or open source their AI research.

crazygringo• 3 years ago

This is so cool. Not earth-shattering or productivity-enhancing, but still really cool.

I could definitely see this becoming a standard feature on desktop and phone wallpapers.

Could also see it being applied selectively to photos in things like historical documentaries -- especially if it can handle the gentle movement of water and clouds as well.

FrostKiwi• 3 years ago

That's one step away from Harry Potter style photo frames for static photos.

mdrzn• 3 years ago

Wow! This seems surreal, can't wait to have it integrated into Photoshop.

Generative Image Dynamics

Related Stories