Ovi: Twin backbone cross-modal fusion for audio-video generation

by montyanderson • 9 months ago

314 114 9 months ago

marstall• 9 months ago

mindblowing - but still in the uncanny valley. and I guess it's cute that many of the characters live in a world where AI has caused an apocolypse, but is that really the message they want to lead with?

christophilus• 9 months ago

To be fair, a lot of movies in theaters have uncanny valleys. A lot of CGI feels that way to me.

alexpotato• 9 months ago

The limits of CGI have gotten so good that if you are noticing the CGI, it's b/c someone skimped on the budget for the particular movie where you saw the uncanny valley.

(Of course, excluding the obvious "that guy just knocked down a building!" CGI)

doawoo• 9 months ago

but still in the uncanny valley

that and the guitar player behind the singer in the concert example has three arms :)

aitchnyu• 9 months ago

I watched a few animes which embraced their poor animation and the excellent Dandadan doesnt chase FPS and resolution benchmarks. Wonder if AI video makers will restrain themselves from getting close to the uncanny valley.

Dandadan intro and its lack of FPS and sharp lines: https://www.youtube.com/watch?v=a4na2opArGY

aspenmayer• 9 months ago

Dandadan is as far as you can get from “poor animation” but it definitely has a specific mix of lo-fi aesthetic animated exceedingly well, and they’re using Adobe Animate, aka Flash.

Animation doesn’t feel fast if it’s too many FPS or too steady, anyway, ironically and counterintuitively. You can’t do everything on the ones and twos.

To your point about Dandadan’s intro, it’s jam packed with references, which is another kind of skill in and of itself:

https://www.youtube.com/watch?v=5sUaK0xahBU

Chainsaw Man is in that same vein, and is another Science Saru production. I’m looking forward to seeing what they will do with the Ghost in the Shell franchise next year.

https://en.wikipedia.org/wiki/Science_Saru

I get what you mean though regarding Dandadan’s animation style; it has a very hand drawn manga vibe, and the detail is minimal yet finely balanced against the overwhelming amount of noise and visuals. It’s like a slapdash superflat.

https://en.wikipedia.org/wiki/Superflat

On a side note, as an anime fan, MBS is doing great work lately. I liked Witch Watch much more than I expected to, and that’s a much better show than the genres involved would lead one to expect.

https://en.wikipedia.org/wiki/Mainichi_Broadcasting_System

meonkeys• 9 months ago

Lazyweb: Are these related? If so, how?

https://news.ycombinator.com/item?id=45603435

https://news.ycombinator.com/item?id=45652726

pavlov3• 9 months ago

When a new open weights AI model comes out, opportunists register a domain using its name and start hosting it hoping to make a buck with SEO.

Easier than ever now, as AI-assisted coding tools will build you that generic landing page and basic UI.

imiric4• 9 months ago

If they're actually hosting it and providing a service, that wouldn't be a bad thing. Ovi's Apache license allows commercial use. They would only be infringing on the license by not publishing it and the original copyright, and would be morally bankrupt by not disclosing the original project and passing it off as their own.

But I also suspect that most of these are indeed SEO scammers, that there's no actual service, and that all payments are pocketed. It might take a few days for the scam to be reported and the site taken down, but it's likely enough to get a few hundred bucks out of it. They'll never be pursued because of where they live, and they can have many of these up in no time, thanks to AI, as you say.

What a sad state of affairs that no "AI" company or government is taking seriously.

pavlov3• 9 months ago

Seems like the video model is based on Wan2.2.

Lots of activity around Wan lately. It’s nice to see flexible open models make a strong showing against the massively funded closed competitors like OpenAI and Runway.

cubefox• 9 months ago

And Google.

echelon• 9 months ago

OpenAI wowed the world with a video model that was also a script writing, editing, dubbing, foley, and music model.

Kling still has the best proprietary video model, but Sora 2 is so smart that you don't need to edit anything if your target is social.

I don't see how Runway, Pika, or the rest of the purely foundation video model startups survive against the giants and the incredible open source Chinese models. They've got to be sweating bullets right now.

Everyone's also sleeping on xAI's high quality and insanely fast video model (10 second generations) that they're giving away completely for free without watermarks.

dang• 9 months ago

Discussed here:

Wan – Open-source alternative to VEO 3 - https://news.ycombinator.com/item?id=44928997 - Aug 2025 (38 comments)

drak0n1c• 9 months ago

They're the main open source privacy-preserving video models that VeniceAI has started offering with a convenient frontend. Ovi is one with image to video and there is also Wan 2.1 image to video, Wan 2.2 text to video. Wan 2.5 is available but it's routed in an anonymized fashion to the official provider. They're also much more affordable compared to the routed Kling, Veo, and Sora options.

tootie• 9 months ago

Kinda terrifying. And it can run in 32GB of VRAM? Anyone with a 5090 can start spewing out believable fake videos.

aussieguy1234• 9 months ago

The other option is to rent a 5090 in the cloud. Probably less than 0.50 per hour at most providers.

latchkey3• 9 months ago

If people are interested, I could split up each of our MI300x into 4 and then charge $0.50 (1/4th our current rate). You'd get 48GB of vram instead of 32GB and it would be HBM3 instead of GDDR7 (5.3TB/s vs 1.7TB/s).

The only catch is that I'd need to get 32 people who want VMs like this since I would have to do it for the entire box of compute.

Wan2.2 runs just fine on AMD.

jonplackett3• 9 months ago

Yeah and even with the server it’s really cheap - things like omnihuman are I think better but MUCH more expensive to run

amelius• 9 months ago

How long until we see blockbuster movies produced by a guy in his basement for <$1000?

computerphage2• 9 months ago

I think "blockbuster movie" is a moving target, so it's a bit hard to know

nine_k2• 9 months ago

It's a relatively well defined measure of success though: a movie which is popular and high-grossing.

ivape2• 9 months ago

Soon as the video models can keep characters consistent across scenes. It could take months of prompting to get each scene, but regular movies have long shooting timelines too. If we ever get to instant movie, thought to scene, then movies will die since people will just daydream through the AI.

hadlock• 9 months ago

I can see soap-opera-style, video-manga becoming a thing, where you get 5x20 minute episodes a week, and an ai-generated 30 min super cut of the week's "events" every saturday. What if DragonBallZ, but new episodes drop every morning before your morning commute? And for $30 a month you can do a choose your own adventure where at the end of each episode, the story splits off and you get an alternate history version, but conveniently rejoins the main story by the end of the week.

Three years ago we had a live streaming autogen-seinfeld twitch stream; some kind of coherent story telling via AI doesn't seem beyond reach today, the tools just haven't fully matured yet.

jayd162• 9 months ago

People can daydream for free now.

neom• 9 months ago

As with many things of human ideation and creation: If they can't understand distribution, a long time.

roarcher14• 9 months ago

Never. I've seen people instantly go from liking a static image to disliking it upon learning it was AI generated. The same applies to other kinds of media. No matter how "good" it is, knowing that it was created by an unfeeling algorithm ruins it for most people.

danielbln• 9 months ago

Never is a long time. The resistance will erode.

jonwinstanley• 9 months ago

This is defo true until the moment it gets so good people can't tell.

jonplackett3• 9 months ago

I read a while ago that big scientific ideas take about 50 years to be accepted. Which basically means they are never accepted. The people who disagree just get old and die.

Younger generation who grow up with AI will just think it’s normal, like we think being connected to the internet via a rectangle you keep in your pocket is normal.

jcims• 9 months ago

New things will be possible that aren’t today. You’ll be able to pick the stars in your movie. The home base can be your childhood home. Your unrequited love can be virtually fulfilled. Etc etc.

imiric4• 9 months ago

"Never" is quite short-sighted.

Most people would use these tools for personal use, if nothing else. Seeing a celebrity, themselves, their friends, etc., act out any scenario they can think of is quite an appealing proposition. And porn, of course, for better or worse.

In the long-term, this has the potential to significantly change how media is created and consumed. Feature films produced by large studios will undoubtedly continue to exist, and they will also leverage the technology, but it's not difficult to imagine a new branch of personalized media becoming popular. The tools are practically already there; they just need to become more accessible, and slightly better.

fart-fart-FART3• 9 months ago

you're seeing them on reddit and twitter. they aren't normal people. normal people don't run a background check on $thing and its creators to determine whether they are allowed to like it or not.

Krasnol• 9 months ago

I don't know how long it will take for photorealistic movies but I'm looking forward to that Hyperion animated movie I'd make myself because nobody wants or is able to.

spogbiper• 9 months ago

honestly i think there is a future in this.. "hey AI model, generate a movie from this book" may indeed be better than letting a director mangle it into something meant to sell tickets. most book -> movie adaptations are not setting a high bar

mvdtnz• 9 months ago

Based on that trailer we're a long, long way away. Ignoring the unsettling facial expressions of the human characters, there is zero visual (or audio) cohesion between the scenes. A full movie of such incoherent visuals would be a difficult watch.

dfxm12• 9 months ago

We'll always be just a few years away.

jayd162• 9 months ago

Unless marketing and the guy's food doesn't count, never. But a "runaway hit for almost nothing"? Not unheard of.

asimovfan• 9 months ago

Probably will depend on how good the writing is.

heavyset_go• 9 months ago

Besides people with weird fetishes, who actually enjoys looking at AI "art"?

nine_k2• 9 months ago

Photos, movies, animation, recorded music, computer games, CGI props in movies, CGI characters in movies were all denounced as "not real art", until good counter-examples appeared.

AI is but a tool; if there is an artist using them, real art can be created, as with any other tool.

ToValueFunfetti• 9 months ago

I enjoyed just about everything in here: https://andymasley.substack.com/p/a-ton-of-ai-images-ive-mad...

aenvoker• 9 months ago

I rather enjoyed some of these

https://reddit.com/r/singularity/comments/1lq299r/postscarci...

https://reddit.com/r/midjourney/comments/1o6ickx/dreaming_on...

https://reddit.com/r/midjourney/comments/1n6mzig/how_to_buil...

https://reddit.com/r/aivideo/comments/1nwdjdn/the_perfect_bo...

https://reddit.com/r/aivideo/comments/1m8a9wz/pinkington_rop...

https://reddit.com/r/aivideo/comments/1n52kut/derek_the_agin...

https://reddit.com/r/midjourney/comments/1muwyah/still_here_...

https://reddit.com/r/DefendingAIArt/comments/1mttoi4/my_not_...

_--__--__• 9 months ago

I fully expect that we will see an AI video project that gets to Skibidi Toilet levels of cultural reach within the next two years, but 'blockbluster' implies a level of financial success that is much harder to predict.

jay_kyburz• 9 months ago

I've yet to see continuity working. Small clips is one thing. To have the same character, wearing the same clothes, revisit environments, with the same lighting and post processing is very different.

I think we'll see AGI first.

jsheard2• 9 months ago

That and control, there's an enormous difference between letting a model just YOLO a prompt, filling in 95% of the details with vaguely plausible whatever, and getting a model to execute on a meticulously planned out shot with every last detail in order.

doctorpangloss• 9 months ago

The main constraint to good movies is good actors and good scripts. So presumably an AI would have to write well or perform well to do that.

dragonwriter4• 9 months ago

How long until we see blockbuster movies produced by a guy in his basement for <$1000?

Probably never. If AI is good enough to cover all the skills needed to do what would currently make a blockbuster movie for less than $1000, the demand for movies will be small enough relative to supply that there will be no such thing as a “blockbuster movie”

xmprt• 9 months ago

I don't believe that consequence. It's never been easier for someone at home to make short videos - see TikTok and YouTube. In fact most people consume most content on those platforms. Yet there's still high demand for movies and blockbuster movies still happen (usually driven by hype on the aforementioned platforms).

On the other hand, I think the quality of movies and expectations will be a lot higher.

dweekly• 9 months ago

The same argument could reasonably be used to explain why no YouTube influencer would ever get more than 1,000 subscribers - if everyone can upload videos that anyone can watch, nobody will really be famous because fame will become very evenly distributed, right?

theendisney2• 9 months ago

Just like with games visual are only part of the formula. In theory you can make a truly fantastic movie (or game) for next to nothing. I didnt believe this before The man from earth. That cost 200K but it didnt have to.

Edit: perhaps 12 angry men was good enough at the time.

imiric4• 9 months ago

It depends on what you mean by "blockbuster movie", as even those can have awful visual effects. But a short film released a few months ago made entirely with video generation tools was surprisingly decent[1]. It still requires a talented "director" to have the right vision to guide the project, but the tools are there.

Before we see this and higher level of quality accessible to enthusiasts, we'll see these tools adopted by mainstream studios first, which is starting to happen.

I'm a firm "AI" skeptic, but if this technology has revolutionized anything, it has been image generation. A few years ago it was science fiction to have the quality of upscaling we take for granted today. I reckon the same will happen with video generation as well a few years from now. Unlike "ASI" and "AGI", these improvements are achievable with better engineering, and don't necessarily require a breakthrough.

[1] https://news.ycombinator.com/item?id=44564697

willyxdjazz• 9 months ago

When the printing press was invented, people didn’t start creating best-selling hits. Humans have always taken it upon ourselves to define what we consider success. Cheap things will always remain cheap…

theendisney2• 9 months ago

Not 1000 but Star Wreck, iron sky and Kung Fury are pretty good.

https://www.energiavfx.com

https://m.youtube.com/watch?v=bS5P_LAqiVg

Im sure more wil follow.

plaguuuuuu• 9 months ago

I'm partial to this music video by Car Bomb, which is AI generated, but somehow manages to be fantastic.

(loud music warning)

https://www.youtube.com/watch?v=1ohaFZllmUE

eichin• 9 months ago

Heh, I used to work for Nokia's Ovi - basically, gsuite for nokia phones (my group did map search) - the official explanation was "Ovi is Finnish for Door", the internal joke was "Ovi is Hungarian for Kindergarten". I couldn't find any backstory about the name here, though.

jillesvangurp• 9 months ago

Same, I worked on several OVI projects in related to places and search. Anyway, long dead, buried, and forgotten of course.

I was in a few of the early meetings on the Helsinki site where I overheard some executives expressing their intention to go after Google. These people had some balls. No clue whatsoever unfortunately. But it was the right kind of ballsy move that Nokia could have pulled off with a bit more vision.

The name was more or less a LOL WHUT?! kind of thing and it flopped horribly with consumers. But still there was some nice stuff in there that wasn't half bad. It's just that the whole branding and rudderless direction doomed it. And of course it was all tied to a failing device software strategy. So when that failed the rest failed as well. I'm not even sure when they pulled the plug on OVI exactly. It was such a non event in the grand scheme of things (mass layoffs, sale of the phone division to MS and subsequent closure, etc.) Must have been around 2013ish I would say. I was gone by then.

pavlov3• 9 months ago

I think the Microsoft ecosystem completely replaced Ovi on the Lumia phones, so it was left lingering on the old devices only.

crorella• 9 months ago

At this rate, in a few months we will have probably some high quality shorts entirely generated by this.

hmokiguess• 9 months ago

It's funny you mention this, I was just thinking this other day we may eventually be in a future where a group hangout party could look like this:

1. Goes to friends' place 2. Usual drinks, whatever gets you going activity 3. Each person writes a prompt 4. Chain them together 5. Watch the resulting movie together

That sounds hilarious and I can't wait to try

epiccoleman2• 9 months ago

I'm vaguely reminded of the excellent Jackbox game Tee Fury, in which players submit slogans for T shirts and "art" separately. Players then get to choose from a few options for slogans and designs to make T shirts which are voted on by the group.

I have fond memories of laughing until I was in tears when playing with a group of friends over drinks during the lockdowns in 2020. Something about the process just naturally results in hilarity (especially if you're in a group where you can be offensive).

It's like exquisite corpse for t-shirts. Or, in your case, shorts.

MangoToupe• 9 months ago

We'll see. I think we'll see a high quality feature film first though, shorts are notoriously difficult to pull off.

nikitalita• 9 months ago

In a few months, we'll have some high quality deep-fakes used to ruin people's personal lives.

CosmicShadow• 9 months ago

This is really amazing, I've been working with AI generation for months and it's amazing how fast separate tools are coming together into one and are usable on your own local machine.

I've been using Ovi for about a week and it's a blast. Like all AI gen, it's a slot machine and even putting in good inputs might lead to bad outputs, but if you run it enough you'll get something good or usable.

I've definitely made many things that look and sound real with both I2V and T2V, albeit T2V tends to look more like 90s tv quality at times, but that also makes it seem more real. If you use Flux SPRO as the image source you can get some pretty realistic looking videos.

I do have a 5090, so it takes about 4 to 5 minutes to make a 5 second clip.

wellthisisgreat• 9 months ago

I do have a 5090, so it takes about 4 to 5 minutes to make a 5 second clip.

what is your setup? took 2 hours for me on 9950x3d with 5090. Any idea what I could be missing? or maybe some other variable is off - i was using default .yml values.

causal3• 9 months ago

Interesting development, terrible company to have it. I cannot think of a company that has done more to use AI to take advantage of young and lonely people than CAI has.

fart-fart-FART3• 9 months ago

half the internet is porn. people and companies legally produce and distribute videos of real 18 year olds of both sexes doing things like ass-to-mouth on camera for money.

and here you are, clutching pearls about AI girlfriends. lol. lmao.

causal3• 9 months ago

They make money trapping minors and lonely adults in fake relationships with models that push users to get intimate, while harvesting and selling the data. The issue isn't AI girlfriends or porn, it's the unethical business model. If they ever get hacked, there will likely be a lot of suicides. It makes the data harvesting of social media look comparatively innocent.

bigyabai• 9 months ago

You've picked a hell of a time to start moralizing the advancement of technology, my guy.

causal3• 9 months ago

Morality? It's wildly unethical to trap children in fake relationships while harvesting their data.

ineedasername2• 9 months ago

For I2V You can get a reasonable “draft” out of it in 1-2 minutes with nvidia >= 4070 & sufficient VRAM at 440x440 resolutions. T2V… not as much, still need to be doing something close to trained resolutions, but I still see some quality, more sporadic, at other known-good Wan resolutions.

Also this model seems to benefit noticeably from having both Cuda >= 12.8 and Torch >= 2.8, and separately SageAttention over Flash 2. But I have yet to see any cache threshold with Easy or Tea that doesn’t get a bit postmodern.

liqilin1567• 9 months ago

Manga may come to life with this :D

Tade02• 9 months ago

Personally I wouldn't mind a better animated season 3 of One Punch Man.

Tade02• 9 months ago

Perhaps it's my pre-AutoTune ear, but to me the audio still has hints of perfect pitch and companding. For the former it sounds much like Machine Head in the Invincible series:

https://www.youtube.com/watch?v=DME86-QucsA

Great work all around though.

bilekas• 9 months ago

Interesting project in the sense of why not see how it works, I'm just still struggling to see the useful usecases of generative audiovisual content. At the moment it just seems to be a nuisance rather than any benefit.

Ovi: Twin backbone cross-modal fusion for audio-video generation

Related Stories