Goodbye Uncanny Valley [video]

https://vimeo.com/237568588

by shawndumas • 9 years ago

197 103 9 years ago

vimeo.com

SubiculumCode3• 9 years ago

Sure graphics are looking amazing...but I still think we are in the uncanny valley. Especially when it comes to faces.

arianvanp• 9 years ago

I agree. Rogue one was a great example. I found it very disturbing. Both Tarkin's and Leia's face

tclancy• 9 years ago

Right and even when it was just about perfect it's unsettling because you get taken out of the moment wondering, "Am I looking at a real person or CGI"?

SubiculumCode3• 9 years ago

It isn't so much the faces look either, but how the faces move. Movement is often too slow, too long, too regular...and ill-timed.

djrogers• 9 years ago

I went in to the movie not knowing that Tarkin was CGI - I assumed he was a stand-in and makeup - and came away none the wiser. After I was told he was CG, it was obvious and weird...

Wonder how much of the uncanny valley stuff left is in our heads?

koonsolo• 9 years ago

Me too, but a friend of mine hadn't noticed that certain characters were CGI, so it must work for some of us.

dschep• 9 years ago

Agreed. And I always associated the uncanny valley with representations of humans, not terrain, robots, etc.

bitwize• 9 years ago

I urge you to watch Blade Runner 2049. Minor spoiler: there is a scene involving Rachael. They brought in the original actress and de-aged her face digitally to look like she did in her 20s and the effect was... mind-bogglingly close to perfect. Easily the best facial CG I've seen so far.

mrec• 9 years ago

They brought in the original actress and de-aged her face digitally

No, Sean Young (the original Rachael) was brought in to consult, but the CGI face was superimposed on a stand-in, Loren Peta [1].

I saw the movie on Monday and wasn't impressed with the quality there; worse than Rogue One Tarkin and on a par with Rogue One Leia. As many others have opined in this thread, the technology's fine for stills but really, really not there for animation.

YMMV, obviously. I have a friend who made it about a third of the way through Zemeckis' 2007 Beowulf before asking "hey, why did it all just go CGI?", and that movie wasn't remotely aiming for photorealism.

[1] https://en.wikipedia.org/wiki/Blade_Runner_2049#Cast

bsenftner4• 9 years ago

It may be a while. We have a large amount of evolutionary neural instinct when it comes to human facial recognition. Realize that camouflage in nature is very common, and as a result our ability to distinguish a real human face from a predator imposter has our entire evolution and survival baked in.

DougN72• 9 years ago

I've discovered something akin to the uncanny valley - 4K video. It looks so sharp and clear it feels artificial. Not sure how long it will take to get used to.

jinushaun3• 9 years ago

Agree. 4K video + high FPS makes real life look 3D graphics.

SurrealSoul• 9 years ago

I think we just need more "real life" events in 4k video and high fps. Right now the only thing associated with that is 3d graphics in video games

fenwick673• 9 years ago

60FPS movies do this to me - everything tends to look like a stage play.

marzell• 9 years ago

Actual movies in 60fps, or movies in ~24fps on a TV that is using interpolation to produce 60fps output? The interpolated/faked experience is often known as Soap Opera Effect [1]. I really dislike interpolated video, but really do enjoy 'real' 60fps video.

[1] https://en.wikipedia.org/wiki/Motion_interpolation#Soap_oper...

noam87• 9 years ago

I've noticed it too with a friend's TV. Some shows / movies become unwatchable because it's so distracting. -- I think part of it is that many of the artificialities of the screen become too obvious: the lighting, the makeup, the fact that many elements on a set a plastic / CG, oddities in perspective, etc... all of these elements are "fuzzed up" by a lower resolution and FPS, forcing your brain to fill in the gaps -- ironically more convincingly.

Sometimes in art, less really is more. Many modern Sci-Fi flicks fail to achieve that feeling of wonder precisely because they are shooting for too much realism in their special effects, I think. The problem is not CGI itself, but the technology does seem to make it too tempting for the filmmakers to fill in blanks that are perhaps better left up to the viewers' imagination.

DougN72• 9 years ago

I think you've hit on it. I feel like I'm on a movie set watching the actors when watching 4K, as opposed to getting into the story like normal (can't remember the term for that...)

jinushaun3• 9 years ago

No. The landscape and nature in that video didn't even look real--never mind people.

ygra• 9 years ago

The pulldown may have helped in that impression, as pretty much every camera motion was jerky ...

adwi• 9 years ago

For those interested and unfamiliar, “pulldown” here refers to converting frame rates between different video standards by doubling up or blending frames at regularly spaced intervals, which can introduce a visual artifact called judder which is apparent in those clips.

https://en.m.wikipedia.org/wiki/Three-two_pull_down

nitrogen• 9 years ago

For some reason the video was uploaded at 25fps which almost never plays right on computers, and frame-by-frame playback shows the skipping is baked into the video. I got bored counting frames to figure out what the source framerate might have been, if there was a consistent source rate.

sgift• 9 years ago

In my experience the moment people know that something is CGI they get into some kind of "extra scrutinize" mode, which moves the goal. A study showing people videos and asking them to point out CGI vs real would be far more interesting to me.

jinushaun3• 9 years ago

Nah. The landscape and plants were too shiny and high contrast. Everything looked plastic.

There were some convincing ones in the video though, such as Avengers and the truck demo.

sp332• 9 years ago

Once you get realistic "rendering" you also need to stick to familiar cinematic styles. There's no motion blur or depth-of-field like there is when shooting with a film camera, so even if the landscape were simulated perfectly, it would still look weird.

y4mi• 9 years ago

Yeah,no.

We're not even close to passing the uncanny valley in movies or games

We might ve passed it in pictures, but that's kinda old news at this point.

two2two• 9 years ago

05:56> as computers graphics get better, we believe all images less

This nugget in the video struck me most. I remember when I began trusting images less once I understood the capabilities modern photo software provided. We've now achieved this with most physical forms in motion picture, short of humans. Which is not to say we won't soon. Once that threshold is crossed, then what's next? Eventually virtual reality, augmented reality, and finally reality reality. We'll trust everything we see (and eventually hear/taste/smell/feel) far less than we do now.

binarymax• 9 years ago

Indeed! I remember this being discussed as early as 1993, in the film "Rising Sun" http://www.imdb.com/title/tt0107969/

Another good example is https://www.cbsnews.com/news/bp-and-the-gulf-oil-spill-misad... ...which I remember being a wake-up call to those who didn't know it had already been going on for a couple decades.

I truly wonder how much of what is presented in the MSM these days is actually not real and manages to slip by...

CM30• 9 years ago

I truly wonder how much of what is presented in the MSM these days is actually not real and manages to slip by...

Quite a lot presumably. Ever since the idea of a media started, there have been people trying to hoax it for whatever reason or not:

http://hoaxes.org/weblog/categories/category/Journalism

If a story is mainstream enough (or remotely political), hoaxes will probably get identified at some point or another. If it's in a specialist subject area? Forget it. Who knows how much questionable stuff slips through when neither the journalist nor their audience know what's plausible and what isn't?

cm2187• 9 years ago

It still requires large enough means that you should know from the context whether a video is likely to have been manipulated or not (based on whether it would be worth it). Unlike a photo where pretty much anyone who can use a computer can photoshop it.

njharman• 9 years ago

50 years ago faking a picture was hard. It won't be long for video to be as easy as still is today.

jamesrcole• 9 years ago

Of course, no one now can know for sure what the consequences will be.... but I'm intrigued by the possibility that this might be a good thing. Could this help make people more savvy about evidence, and less focused on the concrete and superficial? Like I said, I -- we -- don't know.

Perhaps the effect of spam and phishing on the general population provides some insight into this. Have they increased the degree of healthy scepticism and delayed judgement in the population overall? I don't think there's any knee-jerk obvious answer to that, but perhaps there has been some research into that.

siidooloo• 9 years ago

People are situationally sceptical. We tend to be sceptical of things we already don't believe, but not of things we do. We can expect people now and in the future to question an images authenticity if it questions something they want to believe.

mark-r3• 9 years ago

No, we won't trust things less. It's wired into us to believe what we see. Rather we'll get fooled more often.

zebrafish• 9 years ago

eh, i think technology can erase some of the need for human trust. Checksums, public/private keys, blockchain, etc. should all be able to handle verification of authenticity.

As a result, i'm sure we will soon have CAs for more than just https.

bsenftner4• 9 years ago

That mentality right there is dangerous. Yes, it is possible, but in practical execution it will be insecure. I see an active mentality to delegate responsibilities to software that should not be. Case in point: using facial recognition for authentication of services with access to one's finances. If anything, we need to realize the reliance on human trust increases with technology.

csydas• 9 years ago

I had at one time thought something similar, but in review, I think the weakest point is that such things become background noise to the initial reaction and join "References" as the newest proof of validity that never get properly vetted, as we often see with think-pieces that want to take on an air of authority. It's very simple to convince people right now with a reference to an article every other word, regardless of how reliable or even relevant each of the references actually is. The effect though is that the piece attains a state of legitimacy because "boy look at all those references", and disproving such a piece is far more difficult than writing one, with the research to break it down taking far longer, and with attention spans being more interested in whatever the piece is trying to push.

I fear that when we try to use thinks like Checksums, Pub/Private Keys, Blockchain, they're not going to be applied in a reasonable measure, nor will they be explained to audiences in an honest way, but instead simply be tacked on as a badge of legitimacy for the content, not as simply a validation of the source/transaction history.

irrational• 9 years ago

This is how I know the moon landings were fake.

/jk

dcsommer• 9 years ago

Kant and the rest of non-realist philosophers are going to have a field day.

tfha• 9 years ago

Overall a really solid video about the modern state of computer graphics, with a comprehensive tour of the cutting edge realism of light and fluid, the current tricks and limitations used in the modern cutting edge, and then an exploration of the artists and formats that strive to be on the cutting edge of computer graphics while also not aiming for realism.

rdoherty• 9 years ago

Related video about why people think CGI sucks or is still in the uncanny valley: https://www.youtube.com/watch?v=bL6hp8BKB24

Basically we consider CGI bad because we only see bad CGI. If it's used properly and the director/creator understands its limitations we either don't notice it or use our suspension of disbelief and don't care.

hughes• 9 years ago

We have gotten very good at nearly perfectly rendering images that accurately capture the behavior of light and materials, but I feel like to produce truly convincing CGI we need to improve the imperfections that we have come to expect from more traditional media.

As other commenters pointed out here, the opening nature valley scene was unconvincing. I think this is partly due to the author not having modeled the limitations that affect physically filming a similar scene.

- a helicopter or airplane could not plausibly move along the track shown in the opening shot

- camera rigs have imperfect tracking control, sometimes biased along certain axes

- high res 4k sensors have physical limits and defects. the scene is rendered as if captured by a totally perfect sensor

Simulating the motion of a helicopter that doesn't even appear in the shot seems well beyond the effort that most digital artists are willing to go to, but I think these sort of things are key to producing something truly convincing.

pixelbeat__• 9 years ago

That's the point. The opening scene is the uncanny valley and supposed to be not quite realistic

colordrops• 9 years ago

That was my read as well. He even intentionally dropped the frame rate.

Balgair• 9 years ago

In addition to that list, there are the physical issues of film and optical devices. CGI can just work around them, but with film and lenses, physics still gets yah. For reference (http://www.quadibloc.com/science/opt0505.htm) the first order optical issues with all systems are:

Spherical Aberration: this is the aberration affecting rays from a point on the optical axis; because rays from this point going out in different directions pass through different parts of the lens, then, if the lens is spherical, or otherwise not the exact shape needed to bring them all to a focus, then these rays will not all be focused at the same point on the other side of the lens. Essentially, lenses are spherical, they need to be parabolic but that is very expensive.
Coma: this aberration affects rays from points off the optical axis. If spherical aberration is eliminated, different parts of the lens bring rays from the axis to the same focus. But the place where the image of an off-axis point is formed may still change when different parts of the lens are considered. Essentially, your lens may not be flat compared to the film.
Astigmatism: this is another aberration affecting rays from a point off the optical axis. These rays, as they head through the lens to the point in the image where they will be focused, pass through a lens that is, from their perspective, tilted. Even if neither spherical aberration nor coma prevents them from coming to a sharp focus, if we consider the rays of light that are in the plane of the tilt, and the rays of light that are in the plane perpendicular to that, these rays pass through a part of the lens with a different profile. So they may not be focused at the same distance from the lens, even if they do come to a focus in each case. Essentially, lenses aren't perfectly spherical and may have a more cylindrical portion.
Curvature of Field: even when light from every point in the object is brought to a sharp focus, the points at which they are brought into focus might lie on a curved surface instead of a flat plane. Essentially, lenses are round, and film is flat.
Chromatic aberration: Essentially, red light focuses at a different distance than blue. This is the reason for rainbows.

serg_chernata• 9 years ago

I have to say, the most recent Planet of the Apes was seriously impressive. It's the closest I've ever been to admitting that something looks indistinguishable from reality.

However, they are animals and arguably I don't spend nearly as much time looking at apes as I do at human beings. Still renders of humans may look very close but anything in motion still looks completely fake.

dkobran• 9 years ago

Totally agree though the concept of the uncanny valley is based on computer generated or robotic representations of humans specifically -- not other forms of life or inanimate objects. Meaning, not just realism in general.

mark-r3• 9 years ago

The one that did it for me was Disney's remake of The Jungle Book.

sixQuarks• 9 years ago

Some environments pass the uncanny valley in video, namely vehicles, architecture, nature - but complex things like smoke from a large fire are still ridiculously non-realistic.

terminado• 9 years ago

Except half of the modern popular video content discussed in this clip represents evermore uncanny depths of said surreal valley. Overly glossy blockbuster movie FX sequences, subversive glitch gif style viral media, vaporwave, and so on.

So, maybe the title is something like:

  Warm Hugs For the Uncanny Valley And Other CGI Oddities

lifty• 9 years ago

Here is a photo realistic demo video made using Unreal Engine 4.0: https://www.youtube.com/watch?v=E3LtFrMAvQ4

It's quite impressive how close they are getting to photo realistic rendering. Keep in mind, this is done in real time.

olegkikin• 9 years ago

It's not real time, all the lighting is pre-baked, that's why all the scenes are completely static.

I find this quite impressive:

https://www.youtube.com/watch?v=biB7-9qtE_Y

stillkicking• 9 years ago

There's still a huge tell in contemporary CGI, which is pretty obvious when compared side by side to older, practical effects: smoke and light scattering. Given the amount of destructive action we typically get on screen, the picture should quickly devolve into a murky, misty mess that takes ages to settle down. But of course it doesn't, because the audience needs to be able to see the spectacle.

If the scene is composited digitally out of differently rendered layers, the light won't scatter and occlude correctly through the mist. There's a reason why e.g. the original Blade Runner still looks so timelessly atmospheric compared to your run-of-the-mill dystopian cyberpunk city of today.

In that sense, we've been in the age of post-real CGI for much longer than the quest for photorealism suggests. We're not actually seeking photorealism, we're seeking the hyperstimulus version of it that obeys cartoon physics more than real physics. Simulating fire, smoke and fluids is one thing, but simulating realistic combustion, fogging and drainage is quite another.

Then again, you could make exactly the same observation about cars that explode upon flipping over, or the jumping bus scene in Speed. In real life it made a tiny hop and then smashed its front wheels. Magicians don't need CGI either.

Merad• 9 years ago

I don't know about that. CGI effects are everywhere in modern film and TV, but I rarely hear people talking about them except in the case of action scenes or "obviously CGI" (fantasy/fictional) creatures. Take a look at the VFX highlights reel from season 1 of Game of Thrones [1]. Even 8 years ago they were using CGI throughout the show's environments.

[1] https://www.youtube.com/watch?v=YpjdpDx8_l8

bsenftner4• 9 years ago

"Everywhere" is right. Case in point, that old show "Ugly Betty" made major use of virtual sets: https://vimeo.com/19274597

osteele• 9 years ago

The bus scene in Speed used a 50-foot practical jump from a ramp. CGI was used to remove the road that beneath the jump, not to lengthen the jump itself. I guess you could call that a “tiny hop” if you're measuring vertical distance from ground; not if you're measuring distance from road, or length.

Sources: I remember this from Cinefex, which is not online; Wikipedia repeats the description https://en.wikipedia.org/wiki/Speed_(1994_film)#Filming

chasd00• 9 years ago

also, that jump went way further than expected IIRC

barrkel• 9 years ago

Fake physics is the most distracting thing about character CGI in particular. Any time a character is leaping, landing, swinging on a rope, etc. - it all just looks incredibly wrong and jarring.

banku_brougham• 9 years ago

For me the most painful and infuriating example of this was The Hobbit trilogy.

zamber• 9 years ago

Indiana Jones and the Kingdom of the Crystal Skull (2008) http://www.imdb.com/title/tt0367882/ - another example of unreal physics.

Florin_Andrei• 9 years ago

We're not actually seeking photorealism, we're seeking the hyperstimulus version of it that obeys cartoon physics more than real physics.

On a much higher level, this is what art has done since the dawn of time.

Presenting a version of reality that's subtly altered to make a point and convey a message.

Now we can finally do it easily at the low purely graphic level for motion pictures.

Certhas• 9 years ago

Lasers don't make sound in space.

The uncanny valley isn't about making things realistic, it is about making things feel natural. Things flying by make sounds, so it's natural to assume lasers in space zipping by to make sounds. Same with all sorts of visuals rendering situations we are highly unfamiliar with.

falcolas6• 9 years ago

Dear videographers, when you pick a "photorealistic" opening scene, make sure the first major object you see (the palm tree) doesn't look like its made of plastic. It also helps if you don't try and mix close up objects on far away objects (the transition from grass to what appears to be a white-sand delta or a glacier).

Also, if it is a glacier, snow and running water reflect light very differently.

Just three points in the first 10 seconds which puts "the battle for photo-real CGI has been won" on awful shaky ground.

It seems, to me, like photorealism is currently possible for static images, but the moment you add motion, it all goes to hell.

DanBC• 9 years ago

It's a shame you responded only to the title, because the content of the video is much better.

falcolas6• 9 years ago

Time constraints, a rather visceral disagreement with the premise, and bouncing ogres on chairs (so avant-garde!) have postponed completion of the video for me.

khedoros13• 9 years ago

Dear videographers, when you pick a "photorealistic" opening scene

I kind of think that it was supposed to be an "uncanny valley" opening scene. The rest of the video is quite good; it's hard to imagine that they'd flub the opening so badly, unless it was on purpose.

falcolas6• 9 years ago

In my opinion, none of the CGI they showed was really beyond the uncanny valley; most scenes which which weren't still photos or landscapes made predominantly of rock were still quite fake looking. Especially the "beyond" examples of Tarkin and Obama.

bitL• 9 years ago

Awesome video! Thanks for creating it!

ghostcluster• 9 years ago

Unfortunately, a lot of this video essay consists of ideological political speculation from a very narrow perspective. His history of the industry was very handwavey as well, the requisite prelude before political grandstanding.

ch4s3• 9 years ago

prelude before political grandstanding

What are you talking about, and did you watch the whole thing? It seemed to mostly be about the potential future of CGI as art and its place in contemporary/future culture.

frik• 9 years ago

We are still not there. Press pause button on a CGI heavy movie like "2012". The effects work if they are in motion. But looking at them in "pause mode" on 1080p it looks like shit hits the fan. The same for Transformer movies. 90% is real world material with green screen background CGI effects. There are still way too many physical effects going on. And many directors went back to do more physical stunts, because the end CGI-only result was easily spotted and universally hated by the audience.

Jurassic Park 1993 had only 18 minutes of dinosaur screen footage of - 95% of them were miniatures, costumes and hydraulic mechanical robots. Only a very short T-Rex running sequence was rendered with CGI with the predecessor of OpenGL rendered on Silicon Graphics hardware. The good thing is, even today the movie (shot in analog) can be viewed in 2k or even 4k. They will have to re-render "2012" and the Transformer movies for 4k and 8k in near future.

Also the real time rendering is behind, video games still can't render 2005 era movie quality in real time, let alone 2017 era quality.

So a bit pre-mature to call it "goodbye". The progress is great, but there is a road ahead for sure.

brador• 9 years ago

Only at a distance. Failure to extend research into vector textures over bit textures in the 80s now means 3D objects look blurred when close up. A big deal in VR and AR applications.

ttoinou• 9 years ago

I have yet to see a movie that tries to mimic what our eyes really see (with all the weird diffraction effect we have with our pupil and eyebrows, and some particles moving in the liquid in front of our eyes)

curiouslyme• 9 years ago

When I started watching the video, I thought "yep, this clip is a perfect example of uncanny valley" and kept waiting for them to switch to a clip that was actually photorealistic.

Nope, turns out they seem to think the splash video is realistic. The whole video has tons of obvious flaws and overall feels wrong.

I don't know if I have superman vision or what here...

samfriedman• 9 years ago

I was really blown away by one of the video clips this video uses [0]. It's a tech demo for a "zones" effect in VR. Really interesting effect that I could see being a lot of fun in a Sci-Fi game (wormhole travel).

[0] https://www.youtube.com/watch?v=6D-8kkH1KPE

peterashford• 9 years ago

The author doesn't seem aware that "The Uncanny Valley" refers to the inability to make simulated humans appear natural, not making CGI in general look natural. Very annoying seeing someone use a term with a very specific meaning and not understanding it

robclewley• 9 years ago

I agree. The author's claim on the video that it's "common" now to use the term more widely is strange to me. I've never heard of it used in the more wide sense since the term became widespread, and I've worked in computer science research since the '90s. No citation given ...

Goodbye Uncanny Valley [video]

Related Stories