Face2Face: Real-time Face Capture and Reenactment of RGB Videos [video]

https://www.youtube.com/watch?v=ohmajJTcpNk

by benevol • 10 years ago

160 35 10 years ago

youtube.com

mchahn• 10 years ago

Discerning fake video from real just got a lot harder. Video is one of the last bastions of honest evidence.

yeukhon• 10 years ago

I heard light source has been a way to determine whether something is likely to be fake or not.

benevol• 10 years ago

The data vacuuming companies (FB, Google, ad networks, etc.) collect the information about us required to know how to manipulate us and technologies such as this one represent the tools to actually get it done.

sageinventor• 10 years ago

It would be cool to use this to fix movie footage in post production. You could just copy a face over if the actor screwed up

rawnlq2• 10 years ago

Or bringing back dead actors using past footage!

mortenjorck• 10 years ago

This demo is doubly amazing: First, the obviously impressive (and slightly unsettling in its implications) manipulation of the target face, but second, the fact that this is all being done with a single RGB camera.

Consider the massive rig required to perform the at-the-time groundbreaking performance capture for the game L.A. Noire: http://i.kinja-img.com/gawker-media/image/upload/s--y6fmsAIU... This is how far computer vision has come in five years.

rawnlq2• 10 years ago

In terms of gaming applications this could be huge for virtual reality avatars. You can still be anonymous but still convey facial expressions with a webcam!

scoot• 10 years ago

I'm curious how you think a webcam will be able to read your facial expression when you have a VR headset on you face.

NeonVice• 10 years ago

Conan could use this for his fake celebrity interviews instead of just cutting the mouth out of an image. :)

imaginenore• 10 years ago

Hilarious and scary at the same time. The admissibility of videos in courts is becoming more and more questionable.

deelowe• 10 years ago

Wow.

oliyoung• 10 years ago

Terrifying, amazing but terrifying.

vmp• 10 years ago

This is insanely awesome. Something that comes to my mind is the use of this for dubbing movies and TV series; I'm very sensitive about correctly syncing what's being said to what we see, to the point where I only watch movies in their native tongue - even if I don't know the language and need subtitles. This could be a game-changer.

ghayes• 10 years ago

It's even cooler since you could even do this retroactively if you had the original footage of the dubbing voice actor.

Clever321• 10 years ago

I'm curious, how do you both read subtitles and watch that the sound is properly synced to an actor's lips? I can't read that fast, so I spend 80% of my time "watching a movie" simply reading text on the bottom of the screen.

peteretep• 10 years ago

    > I can't read that fast

I can't speak for the op, but I can read a great deal faster than most people speak.

0x4a42• 10 years ago

Subtitles aren't synched to actor's lips. He talked about dubbed voices, not subtitles. :)

JoshTriplett• 10 years ago

Something that comes to my mind is the use of this for dubbing movies and TV series

One of the references towards the end of the video mentions using this for translation, so that's definitely one of the intended applications.

spriggan3• 10 years ago

The video made me feel very uncomfortable, and it takes a lot to make me feel that way.

tibbon• 10 years ago

The paper is great, but I wanna see some source code!

Hydraulix989• 10 years ago

I don't think they want the world to see some source code, should it get into the wrong hands, though the repercussions of this work are bound to hit us sometime. [1]

[1] https://www.youtube.com/watch?v=GBkT19uH2RQ

albertzeyer• 10 years ago

Where can I find the paper?

drawkbox• 10 years ago

Very well done. The best part is how they re-enact the mouth/teeth to look so real by capturing it by sampling earlier parts in the video to then use that on the non expression still or loop. I was blown away when Trump's teeth looked so real then they explained this process and why.

This could be huge (yuuuge) in games and virtual spaces. At GDC Unreal 4 has a demo recently and seems we are approaching that era[1]

[1] https://youtu.be/JbQSpfWUs4I?t=6m

namelezz• 10 years ago

This is impressive.

xchip• 10 years ago

Does anyone have the link to the paper?

mccappy2• 10 years ago

http://www.graphics.stanford.edu/~niessner/thies2016face.htm...

mccappy2• 10 years ago

http://www.ieee.org/conferences_events/conferences/conferenc...

diskcat• 10 years ago

The title is really underwhelming compared to how cool the demo is.

zaro• 10 years ago

Faking news got an order of magnitude easier :)

listic• 10 years ago

What is Target Actor and what is Reenactment Result? The former looks better to me.

izym• 10 years ago

Taget actor is the source material that they're changing, and the latter is the result of that.

kristiandupont• 10 years ago

One of the barriers existing in webcam meetings is the inability to make eye contact. It seems subtle but I think it is more important than one might intuitively think.

I've thought a lot about how to overcome this and came up with nothing but cameras beneath the screen (which Apple seems to have worked on but we have yet to see it: http://appleinsider.com/articles/09/01/08/apple_files_patent...). This technology could possibly provide a competing solution.

SergeyHack• 10 years ago

Imagine live edit of your video conversations, that attaches joyful emotions to any mention of an advertised brand.

Face2Face: Real-time Face Capture and Reenactment of RGB Videos [video]

Related Stories