The Problem With Photorealism

http://blackhole12.blogspot.com/2014/03/the-problem-with-photorealism.html

by blackhole • 12 years ago

163 120 12 years ago

blackhole12.blogspot.com

Minecraft is an interesting case study here, in that it deliberately throws away realistic graphics in favor of a comprehensively mutable environment. It's interesting how that seemed like such a totally unprecedented idea at the time.

danso• 12 years ago

I'm not sure where Minecraft came in the recent era of gaming...since it was built in 2009, that was well after games started being big on iOS, and some of those big games went for 8-bit as a way to maintain a modern sense of style and artistic quirk with the limited dev and system resources. Also, notch got some inspiration from Dwarf Fortress, which made an even more extreme sacrifice of graphics (the precursor to Dwarf Fortress was a 3D game that was too difficult to develop for).

Either way, who could've guessed back in 2004ish that pixelated retro games (including FTL, TowerFall, and countless iOS hits) would be what gaming would look like 10 years later?

lazugod• 12 years ago

Either way, who could've guessed back in 2004ish that pixelated retro games (including FTL, TowerFall, and countless iOS hits) would be what gaming would look like 10 years later?

Flash games were probably the 2004-era precedent.

AJ007• 12 years ago

It wasn't unprecedented, just look at Dwarf Fortress. Tarn threw out all graphics in exchange for focusing on an incredibly complex and deeply layered game -- with an environment that is fully constructable and has both a geological and cultural history.

Once the 3D accelerator become an item all gamers had, (perceptually) it appeared all major game develop shifted to producing "photorealistic" graphics. We ended up with an endless trail of games that basically produced the same thing.

I wonder, what would happen if the budget of something like Call of Duty was spent on a game with Minecraft or even Ascii graphics? Perhaps a total disaster, but what could be produced if it was done really well?

dasil003• 12 years ago

The current state of Dwarf Fortress probably is within an order of magnitude of the number man hours in it as an early Call of Duty. The difference is that the work on Call of Duty is mostly art which is trivially parallelizable, whereas the work on Dwarf Fortress is the engine which is not parallelizable to anywhere near the same degree.

mnw21cam• 12 years ago

Once upon a time there was an open source game called "Cube" which was a first person shooter. It allowed in-game multi-player level editing, with all the occlusion and lighting recomputed on the spot.

It was superseded by Sauerbraten, which improved the level structure to allow multiple floors, but didn't run as quickly due to the calculations being a bit harder.

andrey-p• 12 years ago

A game with very stylised graphics will always win over a game with very realistic graphics as far as I'm concerned. If you've got a game with unassuming graphics it's a lot easier to focus on the gameplay.

Alternatively, stylising things _just right_ can get you a lot further in creating a specific atmosphere - games like Limbo work because everything fits together so perfectly.

coldcode• 12 years ago

Having worked on a FPS MMO, I can't agree with this enough. People always complained it didn't look real enough. I always wanted to ask them if would mind buying 1,000 video cards up front and then wait 10 years buying 1000 new ones every year until I got it right. I figured by then I could retire and duck the problem. Reality is ridiculously hard to do right. Now try doing it over the internet with a half second ping time and you realize that not only reality bites, but physics bites harder.

chiph• 12 years ago

While WoW looks the way it does partly because of it's 10 year old origins, it's artwork also communicates to the player: "It's just a game". And once you're far enough away from an imitation of reality, the brain stops trying to fit it into it's model of the world and complaining when it can't.

zeidrich• 12 years ago

Just simulate the entire universe. It requires a computer the size of the universe.

There is one freely available, however, you will have to share it with a few other people. It's barely noticeable though because it's highly parallelizable.

guizzy• 12 years ago

It's rather poorly documented, though.

mikeash2• 12 years ago

And extremely buggy.

jrowen• 12 years ago

Proprietary assholes. Maybe if they just made the source available we wouldn't have to spend countless man-millennia trying to reverse-engineer this thing.

mamcx• 12 years ago

No poorly, is very well documented. The problem is that we run inside the exe, no in the source code!

ctdonath• 12 years ago

Just need some rocks. Well, a lot of them really. http://xkcd.com/505/

gosub• 12 years ago

You could lazily evaluate the universe and save some cpu power.

matt_kantor• 12 years ago

Isn't that only true if you assume locality?

russell• 12 years ago

Actually the problem is quite a bit simpler. You only have to simulate the observable universe, not the part that is outside our light cone, which is probably most of it. And the observable part makes it simpler still. You dont need to simulate planets more than a few hundred light years away, because they are not observable. It gives you time to upgrade your hardware before the next round of astronomical satellites goes up.

z3phyr3• 12 years ago

Its quite hard actually. By simulating the observable universe, I am not implying to draw the objects on screen. Its much more and computational astrophysicists are struggling like hell!

SixSigma5• 12 years ago

It would have to simulate itself, recursively, so probably a computer greater than the size of the universe is required.

Pxtl• 12 years ago

Heh, actually this makes me think that Augmented Reality gaming is the way to go - like, proper real live-action quake overlaid onto RL environments. Then you only need to model the monsters, the environment's already done.

But I don't think we have the VR glasses for that one yet - I mean, Oculus is opaque.

robmcm2• 12 years ago

This guy isn't thinking outside the box. He's trying to solve the problem using techniques from the past.

Obviously the solution isn't to manually make levels, it would be to automate them. Where are EA on flying scanning drones around football stadia in order to save artists thousands of man hours?

They already use these techniques to scan actors faces, I can imagine someone from 10 years ago predicting it would be impossible to script all the movements of lips and eyelids to produce realistic characters.

Disclaimer: I have no idea what I am talking about, but thought my off the cuff theory would have more weight if I wrote like I did.

msandford• 12 years ago

That kind of tech is definitely coming, but the game studios aren't developing it. They'll buy it once it comes out though.

Game studios aren't VCs. A VC buys pieces of HIGHLY speculative companies trying to maximize his/her odds of a 1000:1 exit. Game studios fund a small portfolio of games and try and make a profit on every single one.

The culture in a studio is so completely different it's very unlikely that anyone high enough up at a game studio to have the budget and other resources to try and develop it in-house won't have the risk appetite to actually do so. And the people who have the risk appetite won't have the budget.

chasing• 12 years ago

Disclaimer: I have no idea what I am talking about, but thought my off the cuff theory would have more weight if I wrote like I did.

Ha. I like the cut of your jib.

Gracana• 12 years ago

iRacing scans racetracks, but that's for physical accuracy; it likely increases the time and money involved rather than decreases it.

http://www.iracing.com/track-technology/

pizza2342• 12 years ago

The author's point is more abstract.

Yes, you can certainly work out a few problems, scaling up the computations in some way.

Some problems though, are inherently hard (eg. animations), and it's hard to imagine some breakthrough even in the next decade.

bigd• 12 years ago

That's why they are called breakthroughs! :)

agumonkey2• 12 years ago

I'm puzzled. IIRC a decade ago Endorphin brought hybrid keyframed / ragdoll / physics animation to studios and a few years back on consoles. Am I wrong ?

chongli3• 12 years ago

Scanning and other forms of automatic content generation do not replace manual 3D modeling and texturing; just as cameras did not replace painting and sculpting.

Personally, I'm more excited about procedural generation than any of these other ideas. Programming seems to be the most powerful tool humans have ever devised. Why not apply it to create art?

MartinCron2• 12 years ago

If you could use automated scanning (or other means of observing the real world) combined with procedural content generation, that could be huge.

Buildings that look and feel like buildings because they were procedurally generated with a deep understanding of how buildings actually exist? Yes, please!

dclowd9901• 12 years ago

Have you looked at google maps 3d view lately? That's all procedural, and the emphasis isn't even on photo realism. It's not difficult to conceive of a drone that does scan and take high res images and software that stitches it all together later.

RaphiePS• 12 years ago

That only works if you're doing a game that takes place in real-life locations. If you want a made-up world, like most games do, then scanning won't help at all.

sp332• 12 years ago

Debevec (yes that Debevec) is working on capturing a performance in a way that lets you re-light afterward. https://www.cgsociety.org/index.php/CGSFeatures/CGSFeatureSp...

dragontamer7• 12 years ago

Ironically, procedurally generated levels was a big thing in the early 90s. Castle of the Winds, Rogue, Nethack, and more were 100% dynamically generated.

Truly dynamic agents existed in Rollercoaster Tycoon, where every single person that walked into your theme park would have Nausia ratings, money, and make individual decisions at every point in the game. G-force was measured on every ride, and would effect different people in different ways. The amount of vomit in front of rides would disgust people in differing amounts. Etc. etc.

True AIs, agent theory and dynamic games were the norm in the mid-to-late 90s. But this approach to gaming isn't what people want. The truth of the matter, is that what makes money are these pre-scripted game-movies that sorta-kinda look photorealistic.

In fact, true-AIs have been implemented in games by probably every game development team. When AIs fear for their own life and run away from players (instead of standing and fighting), players tend to get bored. When AIs become so good at the game that they crush the player, players become frustrated and quit.

Believe it or not, few people are willing to put up with realistic and dynamic challenges. There are no game guides, no assistance, no "help menu" when a game is created dynamically. It is hard to share experiences with friends, because everyone is sort-of playing a different game.

azernik• 12 years ago

People didn't turn away from these games because they don't want realism; they turned away because primitive world-generation and AI are not very convincing. Creating a building layout and a story by hand are much closer to reality than whatever Nethack can generate; the only problem is volume (Nethack could create one very unrealistic dungeon for every playthrough).

monkton• 12 years ago

The guy is also ignorant. Speedtree generates all sorts of photo-real foliage for a million games and film/TV CGI, has done for years. Even Forza Motorsport uses Speedtree for trackside stuff. Also, loads of materials are generated procedurally - fur and hair are the obvious examples. For games, MGS V uses lots of procedural stuff in its modelling. http://www.youtube.com/watch?v=QZFR4H4LORU is the GDC 2013 presentation. Has a lot of neat tricks and optimisations for content generation in general, all in search of the photo-real.

The other problem is research. Loads of people are working on dynamics, rendering and materials. Not very many are working on contextual object distribution to detail an interior so it has decent complexity and realism, or ways to procedurally create Skyrim villages. Even less are working on generating complex quests, but when these things get "good enough", it'll be a huge step forward.

wmil• 12 years ago

That's more problematic than you think. Many architects and / or building owners assert intellectual property rights over the building. Movies can be sued if they are used prominently in shots without permission.

So scanning could open devs up to lawsuits.

archagon• 12 years ago

Miguel Cepero of the Procedural World blog said something similar: that with the increasing complexity of AAA games, the only way forward is to procedurally generate much of the content. He talks the talk, too: the engine that he's currently building can algorithmically create varied terrain and architecture while still allowing you to manipulate the game world like in Minecraft. http://procworld.blogspot.com

Personally, I don't necessarily agree. I've had way more fun with simple-looking games over the past few years (Super Mario Galaxy, all the crazy indie platformers, Minecraft, Counter-Strike, etc.) than with any AAA games. I think the way forward for the medium is to focus more on tight and innovative game design rather than shiny graphics and enormous (but ultimately empty) spaces. No procedurally generated world will ever contain the intelligence of a single cleverly-designed level.

On the other hand, I've also had a lot of fun with roguelikes, and I really look forward to the innovations in procedural generation over the next few years.

DanielBMarkham• 12 years ago

Takeaway: the problem isn't photorealism in games, although that's hard enough. The problem is actually having somebody create the zillions of points of detail required to have something to photorealistically render.

So, seems like the technical challenge is to take a small portal, say 64x64 pixels, and then create a procedurally-based photorealistic (including damage, real-world illumination, and a damage model) engine to power it, right?

ZenoArrow• 12 years ago

So, seems like the technical challenge is to take a small portal, say 64x64 pixels, and then create a procedurally-based photorealistic (including damage, real-world illumination, and a damage model) engine to power it, right?

You may be interested in .kkrieger as a proof of concept for this type of engine. The most incredible part is the whole game is less than 100 KiB in size, but then we're talking about coders who are used to working with in 4KiB limits: http://en.wikipedia.org/wiki/.kkrieger

Talking of the demo scene, you may be interested in this 4K demo, another great demonstration of what procedurally generated content can do: http://www.pouet.net/prod.php?which=52938

benched3• 12 years ago

I am a long-time 3D graphicist, and have written numerous software and accelerated renderers. I am really struggling to see the point of this article (it seems trivial and tautological to me), although I'll be glad if it generates some discussion.

This is an area where I feel very strongly that perception is reality, or at least so close to it as to make no difference. The game world is virtual, so the fact that say a rock is not made of zillions of atoms computing quantum chromodynamics doesn't come as a surprise to anybody. Well, that's a shortcut right there. Immediately, the game world is not arbitrarily "realistic".

But I don't think that's what the word "photorealism" usually means. I know I've never used it that way. I use it to mean "feels subjectively real". By that definition, as the OP alludes to, today's most advanced game engines come very close. In my opinion, they come so close, that the rest is gravy.

That's what will be coming in the next decade. More gravy, more cake, more icing. In the next year or so, somebody (Sony? Everquest Landmark?) will start the inevitable development toward Minecraft-style procedural/cellular automata based worlds, combined with increasingly photorealistic rendering.

rntz• 12 years ago

As I see it, the OP is frustrated that people mistake mere graphics (and in particular, realtime raytracing) as the gold standard for "realism" in games, when there are open problems both more challenging and more important that are still unsolved (truly dynamic behavior, scalable content creation).

I don't know about industry, but in academic CS graphics is a huge field; by contrast, my impression the other problems mentioned are rather neglected (though certainly not unknown). However, graphics and games aren't my specialty; I would love to be mistaken.

ilaksh2• 12 years ago

No, actually he dismissed realtime raytracing out of hand on the false premise that it still requires manual artist texture creation.

We have solutions to "truly dynamic behavior", its called a physics engine. Scalable content creation has solutions also, i.e. 3d scanning and procedural generation.

Real time path tracing is where its at these days, not ray tracing anymore.

All of this stuff can go in hardware and it will within a few years. Main thing holding it back is a lack of resources. I.e., you have billions of dollars being spent on tweaking hardware and software for cheaty approximations and manual artistic endeavours rather than on advancing the technologies I mentioned.

resu_nimda• 12 years ago

I am really struggling to see the point of this article (it seems trivial and tautological to me)

That's because you're a veteran of the industry he's talking about. I see this as directed at people less familiar with 3D game programming: "This is a reminder that our simulations are less complex than reality by orders of magnitude, despite what you see while playing Crysis."

I agree that the focus on "photorealism" as it pertains to graphical rendering techniques was a bit off, because he was also talking about animation and kinematics, content generation, and good old processing horsepower (i.e. we already know a lot of math that would make more accurate simulations but it can't be done in real-time). Again, the tone of it feels like he was responding to a more populist sentiment (e.g. "photorealistic" catching on as a buzzword maybe), but I think a better term would have been the more general "digital simulation of real-world phenomena" rather than just "photorealism."

I think it's mostly just a "hey guys, we're a lot farther away than a lay observer might think, here's a few reasons why."

RyanZAG• 12 years ago

so the fact that say a rock is not made of zillions of atoms

But that's the problem. When the player misses the target with a rocket and hits the rock, it gets a strange looking explosion and then the rock is either still there, or simply 'removed'. Sense of realism? Lost.

In an open world game, the player might want to pick up the rock. If you took a shortcut and just made the rock an object that sits on top of the ground plane, when you pick up the rock there is normal grass underneath. If you throw the rock at a building, it bounces off and rolls around stupidly. Sense of realism? Lost.

That's what will be coming in the next decade.

That's what the article is saying - you're deluded into thinking we're close to solving this stuff. There is no way it's coming in the next decade, it is a long way off with a huge number of very difficult problems to solve. This is like the AI debate. AI is always 'just around the corner'. No it isn't, we are nowhere close to even beginning to understand what we actually need to accomplish once the tricks are taken away.

dragontamer7• 12 years ago

Which is why tons of people play "Minecraft", because the voxel technology and ability to deform the environment is much more real than many other games. There are some advancements in this field, but really... at the end of the day... the best "deformable field" is Minecraft still. And it hardly even tries to be realistic.

Fortunately, most gamers don't care about realism to this degree. Many FPS gamers want a tight engine, fun mechanics. Bunny Hopping was left in the Halo series for this reason. Ditto with super-cancels in Fighting Games (tell me if MMA-fighters cancels their jabs into super-attacks).

Realism for the sake of realism is hopeless. Game programmers need to focus upon entertaining the gamer. Some gamers require a degree of realism... but it should never be the primary point of a game.

benched3• 12 years ago

Since this is inherently a moving-goalposts conversation, I am not going to argue with you. I will say that, speaking as a graphics programmer who has been writing advanced renderers for more than 20 years, I do think the sense of your comment is wrong. In my working life alone, I have seen the state of the art progress from 4-color pixelated 2D, to the current Frostbite/Unreal level engines. I believe that already crosses most of the gap from nothing, to photorealism. I am also thinking more about rendering than all the other things people are discussing in this thread. I also do not think the incredible and dramatic progress in rendering can be sensibly compared to the ongoing slow progress in AGI. Again, we all know that with graphics, it's all an approximation. Perhaps you, being of a later generation, have much higher standards. You don't say what you don't think is coming in the next decade, and you can set that bar arbitrarily high, so I don't see this discussion becoming productive.

pizza2342• 12 years ago

The article's point is essentially that the progress on graphics is much slower than it would seem.

This is not so obvious, because looking especially at static frames, it's very easy to think that graphics have done leaps and bounds in the last decade. Well, when I look at the animations of the real gaming experiences, honestly, I think that there has been very little advancement, which is an example mentioned by the author.

I personally doubt that there will be any significant advancement in the next decade, assuming the current advancement rate (which of course, it's hard to predict).

nightski• 12 years ago

Except graphics have improved leaps and bounds in the last decade, even if they do not live up to the expectations of you or the OP.

Detrus• 12 years ago

Well the article says they don't even come close.

If you focus on screenshots and scripted playthroughs then maybe. But if you play a simple 3D physics demo where everything can be destroyed and Battlefield 4 where only 5 buildings can be destroyed, the subjectivity of realism is lost. It feels constricted.

An ugly 3d physics demo with hundreds of boxes becomes about as entertaining and feels subjectively as realistic as BF4, if you ignore screenshots and pay attention only to gameplay.

Believe it or not, shooting hundreds tank and artillery rounds into the environment turns it all into rubble.

A major priority of game studios is marketing. Currently they depend heavily on screenshots. That means they'll constrict environmental destruction to 1 building per level to preserve screenshot fidelity. If they turned down graphics settings and allow physics engines to contribute to the subjective feeling of realism, they're afraid it won't sell enough.

That's an approach some indie games have taken. Naturally it's more efficient to develop. Maybe efficient enough to get rid of $100 million a year teams and be profitable with lower sales.

Geee3• 12 years ago

I don't know if you guys know how fast path-tracing is becoming reality. It just needs a few more years (of GPU advancement) to get rid of the noise, watch: https://www.youtube.com/watch?v=aKqxonOrl4Q

For art, next-generation photorealistic graphics will be 3D scanned and/or procedurally generated. Lots of work still (you can't scan cars or anything mechanical), but graphics could also be reused more often.

ilaksh2• 12 years ago

Yeah this is exactly what I was going to say and I was even about to post a Brigade engine video. LOL.

McClure kind of dismisses "ray tracing" offhand (and by implication/context, similar techniques like path tracing) with the premise that every texture would need to be manual developed by an artist.

The answer to that is as you mention 3d scanning and procedural generation.

There are quite a few path-tracing software (and even one or maybe a few hardware) efforts out there, and many 3d scanning companies.

I believe that the main thing that is holding these technologies back now is just people not knowing that they are realistic technologies, which keeps them from being mainstream. But once things enter the mainstream consciousness of engineers, you get an order of magnitude increase in the number of people working on them, starting with some of the existing working ideas and you start getting much more practical and inexpensive solutions.

I believe that within say 7 years Nvidia and ATI will either acquihire or build hardware themselves that makes real-time path-tracing, procedural generation, and real-time physics, convenient and efficient.

Geee3• 12 years ago

By the way, if you are interested in (realtime) ray tracing technologies, check out this forum: http://ompf2.com/

By procedural/scanned I meant mainly procedural that is based on 3D scanning (scans used for learning set), such as FaceGen http://www.facegen.com/ Lots of other stuff could be made this way, too.

JetSpiegel4• 12 years ago

You don't understand.

The article is arguing in a different level. It says that reality is inherently artistically irrelevant, and this race to photorealism is a wild goose chase. You can't 3D scan Skyrim!

Geee3• 12 years ago

I get that, but path-tracing is realism, there's nothing to add or nothing to remove. It's a full optical simulation. Path-tracing is not about what 'effects' it can do, it's about speed, precision, and accuracy, and nothing else. That answers the technology question the article is talking about.

Another question was art and content. For that I will say that new tools will be developed that combine huge libraries of 3D scanned content and procedural generation. Artists are still needed for 'art direction', which is the part where he decides what general shapes and materials he wants, and basically traditionally models almost everything like houses etc. That is still a huge amount of work for games like Skyrim, but the end result will always be photorealistic, and it should not be the artist's job to care about how it looks anymore.

eslachance• 12 years ago

About the last point about generating content, I think it's extremely likely that this leads us down the path of Voxels and Procedural Generation. Both those techniques together would help us at least get a starting point for an environment that's generated from certain chosen parameters, and then the "least effort" becomes modifying that environment for your game needs.

I mean, if you can have a procedurally generated jungle with a huge mountain in the middle and a river cutting through the jungle, all you need to do really is to cut through the mountain for your secret underground base, create a couple of roads, camps and checkpoints and you've got a better remake of FarCry 1 (yeah I'm extrapolating of course but you get the idea).

aw3c2• 12 years ago

Wasn't there something about people actually prefering non-photorealistic renderings more, at least in Google Earth?

bitwize2• 12 years ago

And that's a big part of why video games suck these days. Creating a photorealistic or cinematic experience is extremely difficult and requires a huge amount of artistic assets, and even then the illusion only holds up under certain tight constraints. So game developers do as much as possible to enforce these constraints all throughout the game.

lttlrck• 12 years ago

Or it's because the games just suck and no amount of photorealism is going to cover the cracks.

wmeredith• 12 years ago

100 times this. If you want to know how important photorealism is to getting your subject involved with the medium, go read a paperback copy of Lord of The Rings. It's pretty lo-fi.

chongli3• 12 years ago

No, I think the parent is right. The games suck because of all the constraints placed on them by the asset pipeline.

ANTSANTS2• 12 years ago

I think there is a connection; "AAA" games suck for a lot of the same reasons that Hollywood films suck:

* The "average joe" gamer/moviegoer expects incredibly lavish productions with photorealism, explosions, over-the-top action, etc. Making a game/movie for this audience is very expensive and very risky: They need to spend a lot, and charge a lot to a huge audience to make up the difference.

* So that developer/studio tries to minimize their risk and sticks to what they know how to make and what they know people will buy: the same two genres over and over and over again (FPS and third person action, or over-the-top action and screwball comedy), lavish "set pieces" and gimmicks that look cool in advertising but don't contribute anything at all to the gameplay/story (like "the one building you can destroy" or the "quick-time event": "press X to watch the main character do something really cool instead of just doing something cool yourself"), big-name actors for promotional value, etc etc. Some aspects like writing quality aren't considered as important and aren't given as much attention. Some things that make a better game/movie would be actively harmful to its commercial success (true novelty, gameplay/plot that takes effort from the audience to understand/appreciate), and so these qualities are avoided.

* Despite being incredibly formulaic and so on, these games/movies are very successful with their target audiences. To add another layer of metaphor, they're the fast food of their mediums: unremarkable and probably bad for you, but highly available and consistent in their quality. You might wish it were better, but you'll probably enjoy yourself on some level regardless. Critics and more sophisticated gamers/moviegoers decry the state of their respective industries at large while temporarily ignoring all of the cool stuff put out by smaller studios and passionate hobbyists.

In the decades before expensive CG raised the expectations of gamers and moviegoers alike, there was always the equivalent of these successful but substanceless creations, but you saw a lot more experimentation coming out of the big studios. The same company could put out a generic space shooter (sorry, shmup fans) today and a unique exploration game tomorrow. It's simply not feasible for those same companies today to dedicate as many resources on niche titles as they do for their blockbusters, and at the same time they've forgotten how to do small productions. It's not the end of the world, because digital distribution and crowdfunding gives the little guys the power to strike out on their own, create niche, experimental productions, and survive, but it is sad to me that these media juggernauts do more to advance the state of the art in computer graphics than they do in gameplay or storytelling. (No offense, computer graphics programmers, I'm fascinated by the field)

mattgreenrocks2• 12 years ago

I noticed the most recent crop of engines (thinking of Frostbite 3 here mostly) do a lot of cheating to run well. This is most apparent on the awful draw distance of terrain decoration. Draw-in is very apparent at even the highest graphics setting.

Worse, despite games like Battlefield having beautiful art assets, it takes very little for the illusion of photorealism to be destroyed; it could be as simple as getting too close to a piece of tall grass and realizing it's actually just a sprite. Or noticing that a flag is always flapping one way, and smoke is drifting the opposite way.

It's unfortunate, but the brain is really good at picking these inconsistencies out. More stylized games seem to age much better; TF2 still looks great.

olouv• 12 years ago

Reminded me this "What Is Real" trailer (https://www.youtube.com/watch?v=4dnw9dWISvw), I welcome any trick that can help the player's immersion.

marcusmolchany• 12 years ago

I've always wondered, given all the issues presented in this article and the expertise required to solve them does $50-$60 dollars properly pay the teams working on these games? Not even just engineers, there are artists and even actors capturing motion of characters and authors writing up scripts it doesn't seem like $60 would cover everyone involved in creating a video game.

SixSigma5• 12 years ago

You can buy LOTR on Blu-Ray for about $20.

beloch2• 12 years ago

It's often been said that the difference between Steven Spielberg and Ed Wood is merely thousands of subtleties. Tiny things add up to either make or break created worlds.

Clipping is one of my biggest pet-peeves with current video game engines. It really destroys the illusion when stuff that looks realistic and solid starts magically clipping through something else that looks realistic and solid. How about when the camera passes too close to something and you see the hollow insides? This is especially jarring when it happens to be a character's head and you see the backside of his/her eyeballs and teeth! Pick any AAA title randomly, make a character, and then look at that character closely while standing in his/her default start-of-the-game gear. The odds are good that you'll see dozens of little bits of his/her gear clipping into itself while that character is engaged in the default standing animation. It's telling that even excellent artists can't solve this problem for the one combination of gear and poses they know the player will see with 100% certainty.

The next big step for computer game realism is going to be material physics. Simulated materials instead of textured polygons and bounding boxes. This is going to be computationally intensive, but the boost to realism is probably going to be far greater than we'll gain from ray-tracing.

angersock2• 12 years ago

"Simulated materials instead of textured polygons and bounding boxes."

What does this mean? Like, what do you envision this changing?

nuclear_eclipse• 12 years ago

I would guess that it would imply that meshes are no longer static, and that they could bend, stretch, and fold based on their properties. In some ways, we already get this in modern games that try their best to simulate cloth in flags and such, but it's mostly still smoke and mirrors.

beloch2• 12 years ago

Think about how a game treats a block of stone right now. It's a hollow cube with textures pasted on it. It's bounding box is pretty easy to calculate, but a complex object that doesn't match it's own bounding box closely will still clip right through it.

Now, imagine if we replaced that single block of stone with a matrix of tiny blocks. Also, imagine that we've made the bounding box of the complex object fit its geometry much more closely. If the same motion that caused clipping before is carried out, we'll detect it. It's going to take a lot more computation, but we will detect it.

Not only will we detect collisions that we wouldn't have before, we'll be able to handle them in a variety of gameplay enhancing ways. If the stone is meant to be strong stuff, we make it's mass high and the bonds between sub-blocks strong so that the complex object will come to a jarring halt when it makes contact instead of passing into the stone. If the stone is meant to be softer, we dial down the bond strength between the sub-blocks a bit so that they can be broken. The object might dig in a bit and leave a noticeable gouge as some of the sub-blocks are separated from the main block. If the stone is meant to be more like dirt, we make the bond strength very low so that a blow will cause a chain-reaction that will cause the whole block to tumble apart. We can change the probability of blocks sticking together to simulate either wet, clumpy earth or dust that's ready to fly off in all directions when it's touched. Deforming the complex object is also an option. It could be similarly be composed of sub-blocks, but with tweakable levels of stickiness/stretchiness holding them together that would allow us to capture things like flexing or bending.

In short, we really oversimplify the physical nature of objects in video games at present. This greatly reduces computation, but it makes the world seem unrealistic in how objects interact. By splitting things into smaller chunks that more closely resemble real matter we can start to simulate real materials, but at the cost of greater computation that isn't devoted directly towards making things pretty. Prying resources away from the graphics engine has long been something developers just don't want to do.

hueving• 12 years ago

Smaller polygons.

vacri• 12 years ago

One of the things missing from 'realistic' games is specular reflection and friends. Little subtleties like that help. One of my favourite moments in the realism wars was that phase where everyone had to have 'realistic' water, and I remember logging into Morrowind for the first time. "The water does look nice, shimmery and shiny"... but it drew attention to the problem that nothing else in the game shined at all, it was all dead flat.

radiorental• 12 years ago

I look at the loading scenes in GTAV and I wish the game looked like that, rather than the 'uncanny valley' that it is.

Borderlands II is a great example of not trying to do something you cant, instead it has great art that you can lose yourself in.

drakaal• 12 years ago

It should be "the problem OF photorealism" not "WITH". We all enjoy photorealism, but it is hard to make. We have a problem OF 3 times 6. Not a problem with 3 times 16.

Any how....

I worked on Flight sims in the mid 90's. We had a problem of Physics back then. So we didn't do real physics, we precalculated the physics based on a number of factors and loaded them in to tables, and if you were between point A and Point B in a table we took the weighted average of the two. It worked well enough.

Animations today do the same thing.

Anti-aliasing is now done based on point of contrast rather than the whole image. This works well enough, you notice the jaggies on white and black borders more than 45% grey next to 65% grey.

This doesn't strike me as a well informed post. Others have pointed out that it is directed at those new to the space, fine, but if that's the audience then explain "uncanny valley" in there somewhere. Where you can't be "right" be far enough off people accept it as fake because it is. We see this all the time in Pixar. We also see it in racing games with "arcade physics".

ANTSANTS2• 12 years ago

I prefer comics and cartoons to live-action movies. It's ironic, but somehow, a primitive smile or frown on a cartoon character's face feels more genuine and "real" to me than a professional actor pretending to be happy or sad. I can take a drawing as the unfiltered expression of the artist, but when I watch a live-action film and see real people carrying out the trappings of fiction, I have a hard time taking it seriously. It feels silly, kinda like how the so called "uncanny valley" effect makes bad CG creepy and not just unconvincing.

I don't think that's a common opinion, given the lasting appeal of television and Hollywood, and the lack of respect comics receive in comparison, but in my mind, if a drawing feels more "real" than an actual human acting in front of a camera, then photorealistic CG doesn't stand a chance.

Xixi• 12 years ago

On the other hand it might just be a matter of getting used to it.

Japanese live-action movies are a textbook definition of overplayed, and few years ago I couldn't stand watching them. It was making me physically uncomfortable. Fast forward a few years and I enjoy watching them...

z3phyr3• 12 years ago

Can someone list the possible technical advancements in gamedev yet to happen?

-> AI programmers are negative towards AI, saying gamers want 'fun' not Intelligence.

-> Graphics programmers say, the graphics will be photorealastic soon. What will graphics programmers do after that?

-> Gameplay, they are focusing more on creating the same style?

The Problem With Photorealism

Related Stories