John Carmack: Script Interpreters Considered Harmful

http://www.codingthewheel.com/game-dev/john-carmack-script-interpreters-considered-harmful

by LiveTheDream • 15 years ago

131 87 15 years ago

ja2ke2• 15 years ago

Gaming "toy user interfaces" like the one pictured lower in the article generally take so much code because, very often, art directed animation which drives those UIs is often actually assembled entirely in script. Comparing it to an early Unix kernel I guess paints a nice visual image, but is probably not a fair comparison.

In game, in-world "toy UI" like that is intended to look like the sort of sci-fi UI you see in movies, but also be interactive. The stuff in movies is made by a motion graphics designer with a copy of Adobe After Effects. Replicating that movie like look, with all the animations and visual effects you see on the big screen, but ALSO making it work as a fully user-aware, state-aware computer interface, isn't cheap, either on an artist man-hours standpoint, or from a lines of code standpoint.

I imagine its expense could be reduced if a lot of the stuff up on screen was boiled down to some kind of binary data - a lot of work is put into optimizing performance of engine-native animation files and effects which are played on the characters and environments of a video game world - but usually UI (including fictional in world "toy UI") isn't considered part of the main tool chain or process, so it ends up either being highly special cased by hand, or very highly interpreted (sometimes running interpreted code through middleware which reinterprets the code again) requiring quite a lot of bloat to look and behave in a way that is presentable and on par with the rest of the game. (Usually at this point in the games industry, people build their UIs in flash, and then run them through middleware like Scaleform, which reinterprets their flash based bitmap and vector artwork as polygonal art, and reinterprets their action script into an engine-savvy language... not efficient for the game, but efficient for a former marketing guy who now wants to do game graphic and UI design, I guess.)

(For what its worth, I agree with Carmack's assessment. Making highly authorable and also highly efficient at runtime, "AAA visual quality" game UI is just a hard problem to solve, and it's a problem which few studios deem worth solving because after so many years of the current system, the benefits of a change have almost become an intangible.)

pagekalisedown3• 15 years ago

UI is a solved-problem from my point of view with middleware like Scaleform.

rdw• 15 years ago

Carmack is specifically talking about Scaleform, though! And he's not alone -- I know a few game developers who hate that thing with a passion because it's so slow that it's introduced noticeable performance problems in AAA games.

Because he's clearly talking about ScaleForm/Flash specifically, though, I don't think that his comments can be correctly generalized to all "script interpreters". It's kind of the equivalent of buying a Maylong Android Tablet and going around saying that Android sucks or the tablet market is dead. You just happen to have picked the worst possible representative.

malkia6• 15 years ago

You would also have to find flash-experienced people willing to work the way console game developers work. They actually might feel a little bit under appreciated for the stuff they do, and not feel very rewarding at work, especially when they have to constraint themselves with memory, cpu, gpu, i/o issues that might not be visible on the desktop.

Also at some point certain integration between the engine and scaleform is needed, and you would have to lose a programmer there supporting it.

We tried it, and the consensus was it's not of use to us. I wasn't the one evaluating it, so I can't tell for sure. But resource budgeting (memory, cpu, video memory (ps3)) is very debated thing in the console game development, and everyone wants more budget for his need (audio, animation, builders, ui, etc.)

We also gave a try of an early flash middleware for Dreamcast for NHL2K2 (2000) - there were only 16mb of RAM, and flash was taking significant amount (no scaleform relation). We actually got some pretty good menus, and for a video sports game you need very detailed UI for rosters, trades, etc. We also got people to do it back then.

But then the problem was during gameplay. While in the main menu we had the memory to pull it off with flash, during gameplay all memory was for vertex buffers, textures, game data, sound, animations, etc. We were thinking of swapping out game data (and reloading later) when the PAUSE button is hit (and the menu appears), so flash would have enough memory - but it turned out to have some bad latency - few to couple of seconds. Ideally PAUSE ON/OFF should take no time (otherwise it's annoying).

Another thing was that we still had to render the rest of the HUD (scores, replays, etc.) not using flash - so we had to support TWO technologies for UI.

Hence we turned our backs on it. It was nice, but not for our game.

ja2ke2• 15 years ago

Scaleform is great at enabling web- and interactive advertising-trained graphic designers to make video game UI art. It's not as great at being efficient, or truly integrated into your game.

forrestthewoods2• 15 years ago

My favorite scaleform bug was when the game suddenly ran at less than 1fps due to a setting that led to the UI having more triangles than there were pixels on screen. Scaleform is good at a lot of things, but it's not even close to a "solved" problem :(

rwmj3• 15 years ago

The real question is: why do people keep writing interpreters, when writing compilers is not hard.

It's not "interpreters vs C++" as Carmack supposedly said, but crappy slow badly implemented user languages vs efficiently implemented user languages vs a guru writing C++. The middle option is the one to go for.

Especially with LLVM which makes compiling your code very easy, and it's got a BSD-ish license, so no problem linking it with your awesome proprietary game.

delinka• 15 years ago

With an interpreter you sort of get an automatic sandbox because your badly implemented user language is handled by a buch of switch-case or if-else statements. Not literally, but the 'user language' is entirely limited within the interpreter and doesn't mingle with the real game code unless you code up that specific interface.

Enter compiled code. Now we have the opportunity to run unknown binary code right within the game's address space. Bad News in many ways. To get the Happy Medium (compile code within the game engine, load and run the resulting executable safely) requires implementing operating system concepts. It's a Big Job.

This leads my thoughts on a slight tangent: why isn't there a Super Awesome Open Source Game Engine that's the heart of all 3D games? SAOSGE could even provide the extensibility framework (compiling, loading), ready for you to expose parts of your games to player programmers.

nostrademons• 15 years ago

If you control the compiler and the language is memory-safe, you get the same guarantees with JIT-compilation that you do with interpretation. All execution runs within the memory boundaries specifically allocated by the runtime system, and any other interaction with the machine must go through the runtime libraries, where you can perform any checks you want.

krakensden• 15 years ago

> why isn't there a Super Awesome Open Source Game Engine that's the heart of all 3D games?

Unreal is the closest thing, but there's still lots of new/strange/innovative stuff going on in rendering, which means that lots of bits are hard to reuse. Rage, for instance, has a pretty nifty virtual-memory for textures scheme, which no one else has really done.

swah• 15 years ago

Not hard comparing to what? Its still pretty hard... even jonesforth is kinda hard to get for most developers.

rwmj3• 15 years ago

I hope no one uses jonesforth [or FORTH] for real development. It's a beautiful thing to learn and I encourage everyone to learn how to write a FORTH, but it's an anti-pattern for real world programming. (Edit: this is not a contradiction)

mathgladiator• 15 years ago

The reality it is the reliance on tools and sucky education. I wrote a compiler for taking a less-sucky C#/C hybrid compiler to PHP (why php? so I could get things done and hack it when needed): See: http://news.ycombinator.com/item?id=226480

Reality: taking it to production was nice, but debugging and other tools are not available. I spent a lot of time making it possible to debug the PHP and trace that back to C#. This was less than idea.

Then IDE support. While I can happy without an IDE, most people can hardly function.

Trying to educate people on it was (a) marketing fail [web devs hate static typing], and (b) too hard for people to grasp (functional programming + new stuff = fail sauce).

malkia6• 15 years ago

Turn-around time is what matters most. (I work on a console game). You need very fast turn-around time for a designer to test his ideas. If something turns slow, later it can be put in a C/C++ function and exported.

Also sand-boxing, but this is not of concern to the actual scriptures, it's to the rest of the team.

And something to easy deal with tasks/threads, or state machines, and such.

Also a language that allows you to declare easy what needs to be kept persistent (goes in the game save), what can go through the network (game-play related feature affecting all in multiplayer), or what can be local "effect".

marijn2• 15 years ago

Both ActionScript and Lua are compiled, one way or another. Unfortunately, not everyone seems to have caught on to this an people are still widely referring to all dynamic languages as interpreted. Dynamic typing, garbage collection, and just-in-time compilation have a cost, which I guess is what Carmack was referring to, but the term 'interpreter' seems poorly applied here.

Scaevolus• 15 years ago

People usually use "compiled" to refer to languages that lower programs into machine code. Lowering the program into a bytecode isn't sufficient.

Bytecode interpreters must be carefully optimized, and still usually have an unavoidable performance hit from their cache usage patterns (e.g. from bytecode being in d-cache).

richcollins2• 15 years ago

The real question is why would you write either when there are perfectly good languages that you can already use. It seems insane to me that Blizzard dumped Lua for their own homebrew scripting language. I wonder what the rationale was.

ootachi• 15 years ago

Ahead-of-time compilation of a dynamic language with LLVM will scarcely be faster than an interpreter. This is a very common misconception. Good performance for dynamic languages requires dynamic techniques -- polymorphic inline caches, tracing, dynamic type inference, and so on.

rwmj3• 15 years ago

So use a static language with type inference instead.

Most people don't know about these things because of the poor / non-existent education in computer science and lack of any recognized qualifications, but that's no excuse. One day this branch of engineering will need to grow up.

robterrell• 15 years ago

Despite the author's characterization, Carmack's quote doesn't say that script interpreters are harmful or evil, but "bad" in the sense that they're not performant enough for Rage.

Lua was certainly fine for, say, Angry Birds, and has clearly been a huge win for Rovio in rapidly porting the game.

petsos• 15 years ago

That's interesting. Do you have any links on how Angry Birds uses Lua?

malkia6• 15 years ago

You can actually see the lua usage by peeking into the .ipa iPhone packages (they are just .zip files). Maybe android and other systems are not much different.

I've seen at least the lua levels described there.

StavrosK• 15 years ago

As an aside, are you from Edessa?

DanielRibeiro• 15 years ago

You can see it on the indie level editor: http://www.badboll.nu/acme/able/

Also on the dissassembled apk (it is compiled lua by the way, so you might need this[1] to see it more clearly).

[1] http://chunkspy.luaforge.net/

6ren• 15 years ago

His bolded quote is script interpreters are bad, from a performance, debugging, development standpoint.

shabble2• 15 years ago

As far back as Quake 1 (in fact, before Quake needed version numbers), there was https://secure.wikimedia.org/wikipedia/en/wiki/QuakeC which was an interpreted language, although later on it got a compiler to turn it into a native dll.

aw3c2• 15 years ago

I am clueless about this stuff but:

It was interpreted but it was compiled. QuakeC was not a scripting language as one would say. You had/have to compile it. The engine has a VM to handle it.

shabble2• 15 years ago

yeah, you're right. I grabbed the first line of the WP article and missed the part about bytecode and progs.dat.

Quite embarrassing, since quakeC was one of the first languages I played around with, and I don't remember the compile step. Then again, I was working from tutorials so it's possible there was just a 'compile & run' batch file or something.

buster• 15 years ago

Makes me wondering with all those WoW-Addons which are Lua, it certainly can't be _that_ bad..

forrestthewoods2• 15 years ago

Sure it can. When I play WoW years ago there were addons that could take up over a gig of ram. The mods that were item database related could often obliterate performance. I had a beefy PC at the time and there were still a few popular mods I couldn't run.

The perceived advantage of something like Lua is that it's easy to write, allows for fast iteration, and designers/artists can use it. The issue is that designers/artists really shouldn't be using it. To write fast code it requires a programmer and if you have a programmer why not just do it in C++ where it will be several times faster?

A common game development pattern is to write the bulk of gameplay code in a scripting language and at the 11th hour in a desperate panic start moving as much of it as possible to C++ until you fit in 360/PS3 memory and are fast enough.

rikthevik• 15 years ago

I think there's something to be said for the ease of patching with a scripting language, though. If you need to update a quest, just ship a few new lua files and it's done - seems much easier than messing with a binary.

slowpoke• 15 years ago

  To write fast code it requires a programmer and if you have 
  a programmer why not just do it in C++ where it will be 
  several times faster?

Why should I as a programmer not profit from extremely fast design/implement/test/fix/repeat iterations? I'm not saying scripting/high level languages like Lua, Python or Ruby should always be preferred. But it's equally wrong to assert that C-oid languages are always the superior choice. It's much more a question of knowing when to use what and why, which in my opinion makes an important part of being a good programmer.

malkia6• 15 years ago

Surely you profiled to see that it was the interpreted language causing the slowdown?

malkia6• 15 years ago

Being a PC game, they might have easy solution waiting for them - LuaJIT. Now Mike Pall is saying that the garbage collector is still the old lua one, so in your case (item database) it might not give much speedup. Then again it might!

dasil003• 15 years ago

Why should it be surprising that performance still is of primary concern to Carmack? If you're pushing the state of the art forward then I think that will always be the case.

WiseWeasel• 15 years ago

Google's cached version (site was down for me): http://webcache.googleusercontent.com/search?q=cache:veFy_mx...

floppydisk2• 15 years ago

I'm surprised no one picked up on this gem, yet.

>". . . but you know one of the big lessons of a big project is you don't want people that aren't really programmers programming, you'll suffer for it!"

Edit: Fixed formatting of quote.

derefr3• 15 years ago

A lot of other companies' games integrate scripting engines to ease level/mod design, because they don't want to ship a compiler toolchain along with the game. Carmack's games, on the other hand, are just eventually open-sourced, and modded "natively."

richcollins2• 15 years ago

scripting languages weren't really designed for large-scale development efforts involving millions of lines of code. They typically lack the code-reuse abstractions and development toolsets

I find that dynamic languages with first class functions provide much better abstractions and opportunities for code-reuse than languages like C and C++.

enneff4• 15 years ago

That paragraph jumped out at me, too.

I agree with the point, but not its reasoning. The reason why scripting (read: dynamic) languages fall apart at scale is largely because they're too flexible. Abstractions that are convenient in the small can create unmanageable complexity in the large (unless you are extremely disciplined).

jrockway6• 15 years ago

scripting languages weren't really designed for large-scale development efforts involving millions of lines of code

Neither was C.

There are languages developed afterwards with large-scale development in mind, but you have to ask, what did they add? True type safety? Nope. True encapsulation? Nope. Automatic resource management? Nope. C++ is C with several times more ways to write horrifyingly unmaintainable code. All big C++ shops have a very long lists of constructs that must never be used and very rigorous code reviews to ensure that you don't use any of those features by accident.

At this point, we see that C++ has one advantage over C, and that's namespaces. Helpful, but not helpful enough for "large-scale software development". The major innovation that C++ brought was waking people up to the reality that a maintainable codebase must be curated with extensive automatic testing and manual code reviews. Anything else leads to epic failure.

But wait, that's easy to fix! Let's invent a new programming language! This time we'll call it Java. It will be like C++ but with all the ways to write bad code removed. No operator overloading! No multiple inheritance.

And it's true that Java helped in a number of ways. But it didn't solve the real problems. There is still no type safety; null is an instance of every class in Java, but you can't call any methods on it, for example. So instead of a segfault, you get a NullPointerException, but all that means is that the source of the error is easier to determine. But you still have to write a lot of tests to make sure that your code handles nulls properly. (This is compounded by laziness in design like writing "loggedInUser = null" instead of writing a subclass of User that indicates it's not logged in.)

Multiple inheritance in C++ was a mess, but Java's solution isn't much better. What's the conceptual difference between abstract classes and interfaces? Abstract classes are non-composable class-parts that contain API, implementation, and state. Interfaces are composable class-parts that contain API. Why are these two separate concepts? Why not have a generic "traits" feature? (The answer is because Java is mostly a copy of C++, and C++ focuses on irrelevant OO features like 4 different levels of member visibility rather than semantic annotations like "if you compose this method into another class, it should run after the method from the class it's being composed into". Of course, when you have these annotations, multiple-inheritance or multiple-trait-application works perfectly -- see CLOS or Moose. But all C++ had was public/protected/private/friend, and so that's all Java has.)

This is getting ramble-y so I'll get to the point. No modern programming language helps you write huge codebases. If you want to be able to maintain millions of lines of code, you are going to need very rigorous standards and millions of lines of automatic tests. It's the only way we know of. All more modern languages did was push us from "we're leaking memory because we forgot to call free()" to "we're leaking memory because it's pretty convenient to keep all these huge objects around".

They certainly haven't helped us write more maintainable large projects. If anything, a dynamic scripting language embedded in your game means that you'll have fewer lines of code and clearer separation between components. But it's not such a win that you won't have to review code or write tests anymore.

floppydisk2• 15 years ago

Out of curiosity, what do you think a language geared towards "big software" projects would/should look like and does it exist, yet? Or would you argue, at some point, we hit a design vs. code problem where choice of language doesn't matter so much as does choice of framework and we push more of the boiler plate code onto the framework?

jrockway6• 15 years ago

I'm not sure a language can solve all the problems.

One I think would help would be the inability to interface with concrete APIs. If you want to have a variable or instance attribute or argument to a function or method, its type must be declared in terms of an interface, not a particular implementation. This way, you never get yourself into the situation where you say, "I don't want to inherit from Foo, I want to implement its interface myself" but can't.

Beyond that, there is only so much a language can do to help you. It's nice to have guaranteed privacy, but if you use it like the Java standard library does, it becomes the worst language feature ever. To prevent misuse of a powerful feature, you need to be smart and you need to have other smart people reading your code.

The "way forward" is to realize that software development is not easy.

enneff4• 15 years ago

"No modern programming language helps you write huge codebases."

Have you used Go? This is one of our design goals. Go is a really simple language. Its features are easy to understand and predictable in use. Go code is also very readable in that you don't need a lot of context to understand a piece of code. There aren't any colossal Go codebases yet, but so far things are looking promising.

masklinn2• 15 years ago

> There aren't any colossal Go codebases yet, but so far things are looking promising.

Not really. Go keeps all the old errors everybody should know are errors (nullable pointers, shared mutable state, raw types, ...) and then proceeds to add new ones, and packages all of that in a less regular syntax just in case there was any chance to get a good language out of the previous clusterfuck.

jrockway6• 15 years ago

I haven't used Go for anything huge, but it does feel like good progress for the future. (In a comment a few pages down in this thread, I say as much.)

In the end though, bad code is mostly due to bad programming and bad process. Go is obscure enough that the bad programmers haven't heard about it yet. Start offering high-paying Go jobs to anyone with a pulse and you'll start seeing why people hate C++ and Java so much. It's not that the language sucks, it's that the programmers using it do.

It always surprised me to hear that Google "got by" on C++ and Java but I lightened up a bit when I was reading some of Android. Normally you open up Java and are immediately stunned by the smell it's emitting, but when I started reading the Android code, this didn't happen. Classes did one thing and delegated to other classes when they needed something done. The methods were small and made sense. Line of code inside methods were in "paragraphs". There were no comments like "// hack around bug in SomeOtherClassIWroteButAmTooLazyToFix". It was clear that it was the work of someone who knew what she was doing.

I guess I knew it was possible, but was never convinced of it by any concrete code. The standard library, for example, is horrifyingly bad.

Ultimately languages can lead you in the right direction or the wrong direction, but which path you take depends on the programmer. Google requires code reviews for nearly every commit, and they get to hire the top 0.01% of programmers. Imagine what the rest of the world is like, without code reviews, testing, or good programmers.

Go isn't going to fix that little problem :)

thesz• 15 years ago

Go is imperative language. You will need a lot of context to understand a piece of code. Sooner or later. ;)

billmcneale3• 15 years ago

IMO, the absence of real exceptions (deferred et al. are not enough) and generics makes Go unsuitable for large scale projects

jwilliams• 15 years ago

From the Article: scripting languages weren't really designed for large-scale development efforts involving millions of lines of code

In this instance Carmark is talking about a specific instance - scripting for high-performance games, that still need to work with the tradeoff of high levels of reliability.

He even mentions tempted by functional languages such as Haskell and Caml. Then he lists a couple of reasons it's not appropriate - performance is one, the learning curve is the other.

He then goes on to say that he'd think about it differently, but that performance is such a dominating factor and issue for them. That's driving the majority of the logic.

So I'd not draw too long a bow to suggest that he's canning scripting languages in general. I think the author is stretching with that statement.

you'll have fewer lines of code and clearer separation between components

I think that's the most important statement. If you have millions of lines of code, the answer to to _write less_... and that's where dynamic & functional languages are so important.

I've worked on large codebases. It's rare that one language can cover the whole domain without some big potholes appearing (when I say cover I mean vertically in terms of low-high level code and horizontally in terms of function). That's where your DSL and dynamic languages some into play.

WalterBright• 15 years ago

The D programming language has a number of features that are strongly oriented towards being able to deal with very large code bases:

1. checkable function purity 2. checkable transitive immutability 3. code can be divided into checkably memory safe code and unsafe code 4. anti-hijacking enforcement 5. modules with closed name spaces 6. memory is thread local by default (shared memory is typed differently) 7. look for non-null pointers in the near future

shin_lao• 15 years ago

So what you're saying is : "difficult problems are difficult to solve whatever language you use"?

bitwize• 15 years ago

Sounds like someone hasn't used Ada.

watmough• 15 years ago

It's refreshing to see such a great communicator. He spoke, off-the-cuff, for over an hour, at a level that seemed pitched to appeal widely.

The best thing, though, was the obvious love and enthusiasm he has for his team and their software.

peteretep• 15 years ago

> but you know one of the big lessons of a big project is you don't want people that aren't really programmers programming, you'll suffer for it!

Take that and shove it up your cross-functional agile team :-D

John Carmack: Script Interpreters Considered Harmful

Related Stories