How Naughty Dog Fit Crash Bandicoot into 2MB of RAM on the PS1

http://www.quora.com/How-did-game-developers-pack-entire-games-into-so-little-memory-twenty-five-years-ago/answer/Dave-Baggett?srid=z9ZA&share=1

by ekianjo • 11 years ago

934 247 11 years ago

quora.com

JonnieCache2• 11 years ago

If you haven't read the "Making Crash Bandicoot" blogposts, you're in for a treat.

http://all-things-andy-gavin.com/2011/02/02/making-crash-ban...

danso2• 11 years ago

I don't know if Dave Baggett's heroic essay (well, as heroic as a game debugging essay can be), "My Hardest Bug Ever" is part of those blogposts, but it's one of the things I now mentally associate with the legacy of Crash Bandicoot (which apparently involved all manners of engineering feats):

http://www.gamasutra.com/blogs/DaveBaggett/20131031/203788/M...

HN discussion here: https://news.ycombinator.com/item?id=6654905 (with participation from the author, dmbaggett https://news.ycombinator.com/user?id=dmbaggett)

wooderson• 11 years ago

Why don't to actually link to the original Quora answer where the gamasutra post was lifted from?

packersville• 11 years ago

If Crash Bandicoot was so hard to squeeze in 2MB, I imagine other guys like Solid Snake (can't remember its name) would be incredibly hard.

timboslice3• 11 years ago

Metal Gear Solid

joesmo2• 11 years ago

Nice read. I really like this part: "But we worried about the camera, dizziness, and the player’s ability to judge depth – more on that later."

It's interesting that they were concerned with dizziness and the camera, concerns which seem to have unfortunately evaporated in most 3d games made since, to their detriment.

kedean2• 11 years ago

ND still seems to be concerned with it. I did not personally play The Last Of Us, but it's cinematic nature (I did watch a movie recut) makes me feel like they put a lot of work into those aspects.

JabavuAdams• 11 years ago

These concerns are back for VR. Also, remember that Crash Bandicoot was made at a time when devs weren't sure how mouse-look should work.

anon4• 11 years ago

I never had issues with dizziness ... until games started using an over-the-shoulder camera. Basically, if the character is too far removed to the side, I stop connecting my input with their movement. It feels like I'm here and there's some random guy right next to my left who mimics my movements and also I'm incorporeal. It's a sort of uncanny valley territory - if the camera is just a bit to the side - no problem; if it's a far-away but centered behind the back - I'm fine; if it sort of tries to hover to the right, but swings around (Brutal Legend does it) - it doesn't bother me. But Gears of War is practically on the edge and I can't play it for long.

AtmaScout• 11 years ago

You're right about being in for a treat. Thanks for the link.

kayoone9• 11 years ago

Pretty amazing what kind of skills game development required back then. One side of me is happy that we have all these great tools today, the other is sad because you hardly use this low level stuff in todays software development world.

We are solving different problems today, but the level of software development skills required for a game that today could be done by a single person in Unity in a few weeks is quite impressive.

fsloth2• 11 years ago

" the other is sad because you hardly use this low level stuff in todays software development world."

I don't think there is a dichtomy. It's always about smartly leveraging available resources. The problem with modern development perhaps is then that there are these tempting high level orthodoxies that often obscure the core matter at hand. Ie. focusing on some pointless candied abstract object interface rather than focusing design efforts on data flow and the datastructures and algorithms used.

The need for low level optimization has most definetly not vanished. When program data fits into local ram the bottleneck moves into cache optimization.

feld3• 11 years ago

Please write this post. I love my pebble time (but really you guys need to fix some stuff in the new OS)

pjmlp11• 11 years ago

Just try to code for mobile or embedded platforms.

Yeah they might get bigger storage every year, but the reason why Google, Apple and Microsoft do talks about package size on their developer conferences is that size is number one reason for people to avoid installing apps, or to chose which one to remove.

Also given how the app life cycle works on mobile platforms, big apps are being killed all the time they go into background.

codesushi426• 11 years ago

Not so sure about this. Most mobile games are developed with Unity. And what appear to be the cutting edge titles are usually using the Unreal engine. There's not much "to the metal" coding going on.

Besides, most mobile games are being played by the casual crowd. Games don't need graphics that push hardware limits to sell.

kayoone9• 11 years ago

That's not really the same though. You are not really limited by RAM in most cases, also you are not limited by slow storage media like CD/DVDs as it was/is with most of the consoles. The reality is, hardly anyone on mobile writes their own engines (a part from some of the big players) because it's just too much of an investment and Unity (or one of the other popular engines) is well optimized and allows much faster development.

spatulon• 11 years ago

Yes, the PS1's 2MB of RAM are pretty generous compared to some embedded platforms.

A typical engine controller ECU in a car might have 256KB of RAM (and maybe 2-4MB of flash).

zelos2• 11 years ago

Embedded yes, but modern mobile is miles away from this. ~1GHz dual core, triple issue out-of-order CPU with 512MB-1GB RAM and maybe 10-20GB storage? That was a reasonable desktop PC not that many years ago.

javajosh2• 11 years ago

There are some abstractions, like TCP sockets, that I don't think anyone would leave behind. But language abstractions like "classes" or even "functions" are certainly worth digging into. (The new lambda feature in Java 8 is an interesting case - you don't have to get into C to see the complexity there, but rather the bytecode, which is a kind of abstract sort of assembly.)

I'm not terribly familiar with the scene, but there are a variety of competitions for fun and art that operate within highly constrained environments. The demoscene[0], and computer golf[1], come to mind (although ironically computer golf is in some sense has to be very high level). There's also the security scene, which is quite bit-fiddly.

It's also the case that Go and particularly Rust are quite low-level system languages at their heart, so are presumably amenable to running in constrained environments.

[0] https://en.wikipedia.org/wiki/Demoscene

[1] E.g. Rhoscript http://rhoscript.com/

kozak2• 11 years ago

Can you suggest some explanation of the lambda feature in byte code?

banachtarski• 11 years ago

Making an (state of the art) engine is no easier. Making a game is only easier because the engines are more available now than they were in the past.

NickPollard• 11 years ago

A good portion of this stuff does still go on in Games Development.

I don't work in games any more, but on the last title I worked on (Forza Horizon, Xbox 360), one of my colleagues engaged in a very similar exercise in order to allow data for the open world to be streamed in quick enough to deal with car (travelling at potentially 150+mph) to drive through the world without having to wait for loads, whilst streaming direct from the DVD (we weren't allowed to install to the HDD).

Given that the world was open and you could drive in pretty much any direction, trying to pre-load the right tiles of the world was difficult, and seek times made it tough to bring stuff in if it wasn't all packed together. However we were almost at the limit of DVD capacity so we couldn't do much duplication of assets to lower the amount of seeking required.

My colleague wrote a system that took about 8 hours overnight (running on a grid) to compute a best attempt at some kind of optimized packing. It did work though!

agumonkey• 11 years ago

Very creative. I love this kind of stories. The first interactive display, made by Sutherland IIRC, had no framebuffer (irrealistic price at that time) so the team had to stream everything to fit into the small buffer they could get. As I understand it, everything was lazy and in sync with the screen 'machinery' so there was no delay even with limited memory capacity. It felt like hardware haskell.

Cthulhu_7• 11 years ago

I was amazed myself that you can drive around at high speed through a busy city in GTA 3 on the PS3 without any noticable stutter and hardly any popups - good work there, too.

fra• 11 years ago

Some day I will write a blog post about the tricks we have to pull off to fit all code & assets - or get animations running at 30FPS - on Pebble.

Trust me when I say this: low level development is alive and well at hardware companies.

fragmede• 11 years ago

Why wait?

Blog posts serve as great marketing these days, from showing off your engineering team's technical prowess, to recruiting others who hope to participate in some really advanced problem solving.

The press also helps maintain your existence in people's minds. With Google entering the market with Android Wear, and of course Apple's entry; a couple of 'this is how cool a Pebble watch is inside' would do well to not let us forget you exist!

kayoone9• 11 years ago

Yeah sure, not saying it's non existent, just a lot less common. A blog post about that would certainly be very interesting :)

vvanders• 11 years ago

Yup, you see quite a few people from game dev make the transition to the embedded space when they decide that working 100hr weeks isn't their idea of fun.

Serow225• 11 years ago

Subscribed! :)

feld3• 11 years ago

Please write this post. I love my pebble time (but really you guys need to fix some stuff in the new OS)

akhilcacharya• 11 years ago

I talked to a Pebble firmware engineer at Bitcamp who told me about the hoops they had to go through. Really neat stuff.

Qantourisc• 11 years ago

The amount of unoptimized crap they generate in a few weeks (in some cases) is quite depressive too I'm afraid. Game developers are wasting more and more CPU cycles, this of course reduces development cost. But it would be nice if they put some effort in making things run fast. And this doesn't only count for games!

golergka• 11 years ago

Making things fast is more important than making them run fast. And it's not about the cost — it's about iterations for the sake of game design.

pbo• 11 years ago

There is a tremendous number of hardware products and industrial systems where the processing is performed on small and cheap components (microcontrollers, digital signal processors).

Of course there exist very complex components in the category of microcontrollers, some of them even offer enough resources to run Linux, but if you stick to the $1-$5 range the specs are very limited.

Here are two examples, the first one costs around $3 and the second one is less than $1.

http://www.ti.com/product/tms320f28027 http://www.atmel.com/devices/attiny85.aspx

I develop on such platforms and even though there is an interesting challenge in programming these tiny processors and optimizing CPU cycles and memory usage all the time, in the long run it becomes quite strenuous because there is only low-level stuff and I miss the expressiveness and flexibility of more abstract languages.

shultays• 11 years ago

I feel a little sad every time someone says "We have much higher CPU power/memory now, those things are unnecessary". Maybe they are right but I feel like this is not the path we should take

JustSomeNobody• 11 years ago

100% agree. And this goes for everything from server side, to games, to mobile to client side web. Some of my favorite web sites have gone from a 1-2 second load time to well over 6 seconds. And then they're sluggish after they load. It's sad.

Cthulhu_7• 11 years ago

It's not, but what is true is that people can focus more on features than on hacks to squeeze the maximum performance out of their system of choice.

jimmaswell• 11 years ago

If we're free to not waste time on tons of little performance hacks anymore, why do them anyway?

kamaal• 11 years ago

>the other is sad because you hardly use this low level stuff in todays software development world.

During my engineering days(Circa 2005) I programmed in 8085 and our professor would give us all kinds of crazy assignments and small projects. That was the first taste of any genuine programming challenge I faced in my life. Immediately post that, programming in C felt a little boring.

Recently I worked on a embedded systems project for which I relived these kind of days. I had to invent all kinds crazy tricks to work around with resource constraints.

Your true creativity skills are kindled when your resources are constrained in all manners. Time or other wise. Unfortunately you can't academically recreate the same experience.

jeremiep2• 11 years ago

We also have pretty amazing hardware today that is surprisingly underused most of the time.

I've seen many Unity projects with the worst code you could imagine still run close to 60 FPS when shipped.

Having an easy entry point means you're also going to get a lot of mediocre programmers using it. They seem productive in the first few weeks of the project but then quickly slow down halt once they start changing the design and end up with massive overtime hours while trying to debug and optimize the resulting mess.

Its made even worse with managers trying pseudo-agile-learned-in-a-two-day-masterclass adding even more procedures and overhead.

So yeah, you can make games today with less development talent than yesterday, it's still going to cost you more than having skilled software engineers and the resulting product will be a fraction of its potential.

ivraatiems• 11 years ago

I've seen many Unity projects with the worst code you could imagine still run close to 60 FPS when shipped.

On the other hand, a lot of the worst Unity dreck is horribly unoptimized and runs incredibly poorly, despite graphical simplicity and no real visual effects. If you tool around on Steam or YouTube you can find tons of examples - look up, for instance, Jim Sterling's "Squirty Play" series. Not every bad game on there has issues, but many do.

[1] https://www.youtube.com/playlist?list=PLlRceUcRZcK0zAt8sV33Z...

mianos• 11 years ago

Multiple orders of magnitude less artwork. Multiple orders of magnitude less polygons in models. This would explain a lot of the size difference.

MojoJolo• 11 years ago

But it also has multiple orders of magnitude less in terms of memory. So they have so little or no room for excess.

ZenoArrow2• 11 years ago

Sounds impressive.

Might also be interested to know that the PS1 version of the first Ridge Racer ran completely from RAM (aside from music tracks): https://en.wikipedia.org/wiki/PlayStation_models#Net_Yaroze

jbrooksuk• 11 years ago

Ridge Racer was such a great game and always felt so smooth to me and now I know why. Thanks for this fact! :)

ZenoArrow2• 11 years ago

You're welcome. :-)

codesushi426• 11 years ago

Pretty awesome. Goes to show the great lengths taken even on state of the art hardware at the time.

And the PS1 wasn't even the worst of it. The Sega Saturn and N64 were both considerably more difficult to develop for. And the PC market was terribly fragmented and had a high rate of obsolescence.

This stuff can give you nightmares: http://koti.kapsi.fi/~antime/sega/docs.html

Grazester• 11 years ago

I sometimes wonder what could have been had Sega released proper documentation and a decent dev kit earlier for their difficult to program for Saturn. It was only later on in its life did Sega start to utilise the second processor. Shenmue was originally being worked on for the Saturn. How's that for ambitious. It was rumoured to require the 4 meg ram cart. That I believe 100%.

https://www.youtube.com/watch?v=foZUcPQAMvg

codesushi426• 11 years ago

Absolutely. I've seen the Shenmue video and it's stunning. It's more impressive than anything that launched on the PS1 or N64.

I sometimes wonder what could have been had Sega released proper documentation and a decent dev kit earlier

That is a complaint I have heard a lot, and it is valid. But ultimately I think Sega shot itself in the face by even having a second CPU. Concurrency is a hard problem, and certainly game devs back in the mid 90s were not up to the task of utilizing a second CPU. They had enough on their hands with transitioning from 2D to 3D already. I have read that most games developed on the Saturn only used one CPU.

Choosing quads over polygons was also a major blunder of the Saturn's design. The list of Sega's mistakes with the Saturn is so lengthy that it's impossible to think it had any chance of succeeding.

But I still play mine. :)

The sad thing is while Sega did everything wrong with the Saturn, they did everything /right/ with the Dreamcast and it still failed miserably. I kind of think of them as the Commodore of the games industry. Technology that was ahead of the curve, awesome products, but ruined by terrible management and stomped out by juggernauts (MS Windows, Sony Playstation).

iNerdier• 11 years ago

Dave Bagget could write a book on the making of crash and I'd buy it in a second. There was something very special that went to making that game and I was just the right age to appreciate it.

gherkin• 11 years ago

Take a look at http://all-things-andy-gavin.com/2011/02/03/making-crash-ban... as mentioned above :)

dmbaggett5• 11 years ago

Thanks!

It's on my bucket list to write that book -- also including many humorous tales from my 10+ years at ITA Software, and anecdotes from my current startup (http://inky.com).

If I live long enough, that is. :)

noir_lord• 11 years ago

"It's behind you" - http://bizzley.imbahost.com/download.html

Biography of the guy who did the R-Type port for the ZX Spectrum, it's a riot of a read and has a lot of the old info.

kozak2• 11 years ago

Old console games are great examples of how creativity benefits from constraints.

Someone1234• 11 years ago

Or that survival bias is a real thing. People only remember the good games that survived obscurity for tens of years, while ignoring the majority of terrible games that didn't.

A lot of people like to claim "old games used to be better!" but pick any month of the 1990s and look at the newest releases for that month, I bet for the average month there might be one title you've even heard of.

Htsthbjig• 11 years ago

I am waiting for Mr. Baggett book on the Lisp internals of Crash Bandicoot.

I don't care about the game, I never liked it, but the Lisp code of this man should be on par with PG's.

dmbaggett5• 11 years ago

That would have to be Andy. I'm really kind of a hack when it comes to lisp programming. :)

But yes, Andy's lisp code is certainly great -- all the more so because he also wrote the lisp compilers that compiled it. :)

e40• 11 years ago

He didn't use a commercial lisp? I thought he used Allegro?

landmark2• 11 years ago

confirmed: I'm a shitty developer

xedarius• 11 years ago

Resources blowing the memory was a common problem on the PS1. A common trick was to bake the resources into the exe. I worked on a game where the final disc was a directory of 25 exe's. Each exe was a level, even the front end was a separate exe. You could see that the size of each file was under 2mb, so you knew it would work. You would never/hardly ever dynamically allocate memory on the PS1.

There was a lot of duplication, but the CD was a huge resource and memory was thin on the ground. Also meant that we could use the CD for audio during the game.

Mahn2• 11 years ago

That was the "easy" way of handling it and most devs resorted to it, but then you couldn't load things dynamically and seamlessly in the middle of a "level".

trumpete• 11 years ago

As mentioned in the answer, there were levels that were a lot bigger than 2 megabytes. What their goal was to ensure that these bigger levels would seamlessly be in the memory at all times

sown• 11 years ago

Time + Internet = A smaller world that doesn't forget

tluyben2• 11 years ago

I still play Crash (2/3, not 1) quite a lot on the OpenPandora & on my PS2 when i'm in my wood cabin. I still like it and that's partly of how much constraints there were on the PS1 to get something 'big' like this out there.

I keep hoping for a book with annotated (including lisp) source... Please please!!

fit2rule• 11 years ago

Crash on OP is a real treat - its also one of my favourite wastes of time from the Pandora repo .. truly wonderful to have it so easily portable!

ekianjo• 11 years ago

Yup, did it on OP not too long ago as well :)

kayoone9• 11 years ago

Looks like the system RAM on playstations increased by the factor of 2ˆ4 / 16x with every generation. 2MB to 32MB (PS2) to 512MB (PS3) to 8GB (PS4)

Jare4• 11 years ago

Imagine the uproar among developers when the 360 was going to have 256MB (from the Xbox's 64MB).

http://www.1up.com/news/epic-games-cost-microsoft-billion

theandrewbailey• 11 years ago

Unfortunately, I can't find a reference (probably in a video), but I heard that Bethesda Softworks had a party when Microsoft announced that the 360 would have 512MB RAM.

userbinator• 11 years ago

Following the same trend, the PS5 having 128GB of RAM seems a little ridiculous now, but consider that the PS4 was released in 2013, and the PS3 in 2006, so perhaps in 2020 it may really have that much.

kayoone9• 11 years ago

Yeah i thought about that too. My PC already had 8GB of RAM in 2009 and even today that is still totally fine for most games. However, consoles usually share RAM between GPU and CPU though and high end PC GPUs today have up to 8GB of VRAM which will only increase once 4K content becomes more common place, so something like 64GB shared RAM for a supposed PS5 does not sound so outlandish as it may seem at first glance.

Cthulhu_7• 11 years ago

I think that by then we'll have solid-state memory or something like that.

M4v3R• 11 years ago

Ultimately Crash fit into the PS1's memory with 4 bytes to spare. Yes, 4 bytes out of 2097152. Good times

Wow. Just wow. One can only imagine the amount of hard work and sweat that was put into making this possible. And the pride of developers when it actually worked and the game has become a success. Great story.

saurenandroid• 11 years ago

Some of the developers nowadays need to take clues from these guys.

escherize• 11 years ago

Care to elaborate?

Sir_Cmpwn• 11 years ago

The story of how Mew (Pokemon) came to be is also pretty interesting. From memory: when they were getting close to shipping, they had a completely full cart (only one or two bytes to spare). They removed all of the debugging stuff, which freed up enough space for a new Pokemon to be defined. A programmer secretly added Mew right before they shipped, and Nintendo didn't find out until a couple of weeks later.

maccard2• 11 years ago

Yep - http://www.nintendo.co.uk/NOE/en_GB/news/iwata/iwata_asks_-_...

Navarr• 11 years ago

That sounds like a rumor. Mew already existed in the storyline unless that programmer then added the entire Mew storyline (inc. Mewtwo) - otherwise there was no reason for Mewtwo to exist

bryanlarsen• 11 years ago

4 bytes to spare isn't surprising. When you're crunching things down, you stop once it fits. I once worked on a project that used 4x of its memory space when the project was half done. When I was done, I had 13 free bits of space. Yes, bits; I was doing a lot of bit picking to get it to work.

What was surprising is the lengths they went to to make things fit. A solver? Wow. My problem was relatively straight forward in comparison: just bit packing and silly amounts of code reuse: Hey, these two completely unrelated routines have the same 7 byte sequence; can I make it common?

Fun times, I miss projects like that.

stackcollision• 11 years ago

Yes, this is kind of like "of course the remote is in the last place you look, why would you keep looking once you found it?"

A little more than a year ago I was working on a very space-constrained device: only 2kb of program flash (an attiny23 for those curious). I had to use libusb with this, which ate up a huge portion of that space. My first shot at the main program put me over the limit by almost 500 bytes. By the time I was done, I had packed the program + the usb lib into the program flash with 4 bytes to spare.

Man, was that fun.

eru2• 11 years ago

I wonder why they didn't use an off-the-shelf bin packing solver. But I guess open source solvers weren't a thing back then, and the commercial ones were way too expensive. (Not too mention that the devs might not have heard of these beasts.)

znpy• 11 years ago

Gloriosus tales of a forgotten past.

gavanwoolery2• 11 years ago

Crash Bandicoot was quite a technical feat, but also (perhaps less talked about) - a marketing feat. The series went to sell ~50 million copies IIRC and was also the best selling Playstation game of all time.

Mahn2• 11 years ago

Games that are well engineered tend to be good games for some reason :-)

gavanwoolery2• 11 years ago

See: Doom :)

jkldotio• 11 years ago

I recall reading that at one point game programmers were using portions of their code as textures. Which reminds me of the story of Mel. http://www.catb.org/jargon/html/story-of-mel.html

TheLoneWolfling• 11 years ago

Now here's a crazy idea:

Stenography via rearranging executable code within a binary file so that the binary file looks most like a particular bitmap.

qrmn3• 11 years ago

Indeed, the old FTL game, Dungeon Master (an old RPG, its most direct modern successor-by-inspiration being Legend of Grimrock) put some copy-protection code in one of its sprites, where it hoped you wouldn't notice.

One part of it would repeatedly read one "weak" sector that had been incorrectly encoded on the disk in such a way that it would usually read inconsistently, and would crash the game with an obscure system error if it didn't read differently.

Cute, but kind of a problem when you have a much better quality floppy drive which read the weak sector consistently.

listic• 11 years ago

I wonder if anyone controls the physical layout of bytes on the disk, at least for things like installing large software packages on an HDD, or a major release of an operating system on a DVD. It doesn't look likely: every time I install Ubuntu, I feel the process could be made much faster.

gambiting8• 11 years ago

We do for PS4 disc images. Basically you want to lay out the data on the disc in such a way that the game can still be played while the system copies everything to the HDD. There are pages upon pages of manuals in PS4 SDK how to do it efficiently.

cnvogel2• 11 years ago

At least for Windows there's now a way to install the OS so that it only occupies one continuous large filesystem image with the compressed install files, and only stores the changes to this frozen filesystem images in the "traditional" way. It's called "wimboot".

http://labalec.fr/erwan/?p=1078

That way installing the OS mainy consists of copying this huge image file, hence the "physical layout of bytes on the disk" will mostly be fixed.

I'd guess that this could also be replicated in Linux, but personally I don't know if this is being done. I usually "debootstrap" or "pacstrap" my installs from a bootable USB stick :-).

caf• 11 years ago

Yes, you could replicate that in Linux using overlayfs, with a readonly mount of the static image as the the lower layer and a writeable upper layer using an ordinary filesystem. It would be interesting to try it.

shubb• 11 years ago

Sure. On one project we built a custom Linux distribution by mounting a file as a loop back file system and copying the right files in. We installed a grub boot sector at the start. Then we copied it as a file into a simple live usb. The live usb also containing a script that used dd to copy the data from that file over the start of the first disk of a system you plugged it into. Nice quick install usb, minimal work.

We were installing into standard hardware so it was pretty good. But if your hardware varies this approach isn't great.

We could also have created a fresh filesystem on the device and untarred into it but we wanted to keep se Linux attributes.

derefr2• 11 years ago

That's effectively how CoreOS works.

jeremiep2• 11 years ago

Ubuntu targets a LOT of different hardware, it would be much harder to optimize the layout on their ISO image than it is to optimize for a fixed hardware platform like consoles.

Cthulhu_7• 11 years ago

Do you still use a CD / DVD to install your OS? That's probably the problem right there; CD's/DVD's are relatively slow. Go for an install off a flash disk (where physical layout shouldn't matter), and / or a web install where only the relevant software compiled for your hardware is downloaded.

yellowapple2• 11 years ago

CDs/DVDs don't have to be slow (or at least not as slow as they typically are). Most speed problems are the same speed problems one encounters with spinny / non-SSD hard drives: namely, that data isn't sequential, requiring a lot of skipping around the disk in order to piece together files.

Web installs are also much slower unless you have a reliable internet connection (which is not a given). Installing from a USB stick (preferably of the 3.0 variety in a 3.0 port) is indeed the best option for all but the oldest of machines (in which case you'll need a boot floppy to start the machine up with a bootloader capable of initializing the boot medium (USB, CD/DVD/BD, network) and booting from it).

goalieca• 11 years ago

One really nice benefit of SSDs is that there is no needle or laser to seek. :)

i_are_smart• 11 years ago

and this had to be paged in and out dynamically, without any "hitches"—loading lags where the frame rate would drop below 30 Hz.

This is what gets me. Modern game development seems to say "eh, a little hitching won't hurt anyone", and then we wind up with games that run like shit. Even on consoles.

speeder• 11 years ago

I was playing recently Axiom Verge...

Great game, except it has NES graphics and stutters like hell.

Wasteland 2 on my machine also ran really, really badly, it was unplayable (it was the first time I got pissed for kickstarting something).

Kerbal Space program also has some performance issues, but not bad as the previous two.

Then I go play some graphics heavy game made by some studio that like to make good tech, or play emulated Wii or PS2 games, and there are no issues and games look awesome.

TillE• 11 years ago

Two of those games use the Unity engine, and Axiom Verge apparently uses MonoGame.

Garbage collection is often a big problem for real-time performance, especially on the old version of Mono that Unity has.

gambiting8• 11 years ago

I work as a programmer in games industry,and I feel like the problem is made worse by artists and level designers who add more stuff without worrying about performance. I can make a super efficient physics system or model loader,but that only means that someone somewhere is going to add more particle effects or lights or whatever, or maybe placing too many props in the scene so PS4/X1 can't handle it. In fact, the separation is huge nowadays - I know our engine inside out, but I personally wouldn't really know how to use the editor to remove things from the scene. Likewise, a level designer will know how to put props in the scene,but they will have no idea how underlying things are wired together or what is the performance cost of doing them.

It's a complicated problem, which might have been made worse by the fact that games are simply easier to make nowadays than ever before.

nthj• 11 years ago

I am not a game developer, just a developer and a gamer, so please forgive what may be a goofy question: have you seen or engineered systems that capped the level designers' resources? For a super simplified example, I think of Forge in the Halo series, and I'm pretty sure Forge had a certain amount of monopoly money that gamer-designers ran out of eventually. Would such systems be infeasible more for political reasons than technical?

talmand2• 11 years ago

Back when I was somewhat involved in the gaming industry a very, very smart programmer told me the reason his game with his new fancy, innovative, advanced engine didn't work out.

If you make it possible for level designers to make six square mile levels, they all make nothing but six square mile levels.

pcwalton• 11 years ago

This is eerily similar to the problems we face in the browser space. Make CSS styling faster and developers will just write more complex CSS to take advantage of it. Then people blame the browser (and the Web in general) for being slow.

The difference is, sadly, that we don't control the assets at all. :(

fosforsvenne• 11 years ago

This is also why improving fuel/power efficacy in industry usually doesn't help the environment.

mattbrewsbytes• 11 years ago

With internet based games I wonder if that loading screen/hitch is essentially finding you a server as you move from one zone to another if the server you were on was too crowded.

I also wonder if they are doing the equivalent in modern consoles - pushing the limits with graphics, etc. where you simply can't avoid the hitch.

It would be good to hear someone's perspective on this that works on these types of games.

Cthulhu_7• 11 years ago

It shouldn't be, networking should always be in the background and for console games there's no reason why clients should move to another zone - loadbalancers direct new clients to available servers, they don't or shouldn't move existing clients to other servers to make room for new clients.

Chances are they're not doing anywhere near the same thing with modern consoles; maybe in off-the-shelf engine code to get maximum FPS, but the games themselves, not likely. Also because modern video games are millions of lines of code - you don't want to duplicate those tenfold by squeezing every bit of performance out of it. Maybe only in the most frequently accessed codepaths.

Retric2• 11 years ago

Moving from one server to another can be instantaneous. Just use the cellphone model where they do proper handoffs between base stations. As to loading assets, plenty of games used a streaming model where you can explore huge worlds without issue.

PS: Skyrim is an interesting case where there is a player made patch to make city's open world where the original game has a loading screen. http://www.nexusmods.com/skyrim/mods/8058

zelos2• 11 years ago

Most Nintendo games still seem to be locked at 60Hz (sometimes 30 maybe?). Look at the fuss when people realised that Mario Kart 8 dropped to 59Hz sometimes ;-)

phire• 11 years ago

Nintendo generally picks a framerate and sticks to it. The N64 zelda games were actually locked at 20hz.

(It is possible to do 60hz on the N64, but it's really hard)

adestefan• 11 years ago

The worse thing that ever happened to game development was the on-demand updates. There is nothing worse than buying a game on release day only to wait for it to download a patch.

timboslice3• 11 years ago

I get the sentiment (they should have tested more thoroughly), but I for one appreciate the on-demand update mechanism. Would you rather play a game with previously unknown bugs, or have them smashed on launch day and get a patch to make your experience more stable?

Lawtonfogle• 11 years ago

It is both good and bad. It allows delivery of critical fixes and new content, but it also decreases the demand for code quality from the start as well as increasing DLC. The bigger issue though is for the people who can't get it, like those whose only options for internet are dail up or satellite (with a 5GB per month limit). Sadly the market doesn't care about this small group enough to matter and we get left behind.

But, even with having a worse experience than a day 1 patch (that being unable to get the day 1 patch because it is 10GB and you only have 5GB for everything for a month), I wouldn't call it the worse thing ever to happen to game development.

pandaman• 11 years ago

Yup, yup. Running into a game breaking bug, bringing the game to the store and being told that you've got a "damaged" disk that they will replace (with a patched game, if it's been already "reved" or with an exactly same disk otherwise) was so much more fun. Miss those days too.

gambiting8• 11 years ago

When we submit games to Sony/MS/Nintendo for publishing, we usually have to do it 2-3 months in advance. What are programmers supposed to do in that time between "end" of development and release date? Of course everyone works on little things that were left, you might as well release them as a patch!

balls1872• 11 years ago

Modern game development seems to say "eh, a little hitching won't hurt anyone"

Or just like in the PS1 days, game developers still have to make trade-offs to meet dead drop dates set by publishers.

Online enabled patches can allow developers to be a little more cavalier with the quality, as they push to build more features closer to ship date.

teacup502• 11 years ago

"Online enabled patches can allow developers to be a little more cavalier ..."

That just means that we get to deal with buggy crap while they tell themselves it's OK because they can ship another update.

Sigh.

hitlin37• 11 years ago

very good read.

Kenji• 11 years ago

(Incidentally, this problem—producing the ideal packing into fixed-sized pages of a set of arbitrarily-sized objects—is NP-complete, and therefore likely impossible to solve optimally in polynomial—i.e., reasonable—time.)

Aren't there polytime algorithms that approximate to a certain percentage of the optimum?

riffraff• 11 years ago

yes[0], the author mentions "first fit" which presumably is "first fit decreasing" which is one of those. His approach was, from what I understand, to try a few approximate techniques and choose the best result without trying to run an exact algorithm with unbounded time.

[0] https://en.wikipedia.org/wiki/Bin_packing_problem#Analysis_o...

dmbaggett5• 11 years ago

Exactly. You said it better than I. :)

dmbaggett5• 11 years ago

Yes, but approximate algorithms for NP-hard problems hadn't been as thoroughly studied back then -- this was 1996, after all -- and even if I'd had the appropriate papers at hand, I probably wouldn't have invested the effort to implement anything complicated. There was just simply too much to do in too little time. The simple approximation using greedy packing did pretty well in practice; I'm sure how close that comes to optimal is well understood by now.

ermias• 11 years ago

Great post! Unfortunately it's on quora

GolfyMcG• 11 years ago

I don't mean this sarcastically but what's wrong with quora? I've always found it to have pretty good content but your comment seems to be implying it's assumed to be a bad place to read things. Why?

gergles• 11 years ago

To read the rest of this comment, please log in or append a secret URL parameter to the URL.

adibchoudhury• 11 years ago

Gotta love Naughty Dog. Still leading the industry! Can't wait for Uncharted 4

leeoniya• 11 years ago

also great read about packing: https://fgiesen.wordpress.com/2012/04/08/metaprogramming-for...

submitted before: https://news.ycombinator.com/item?id=7739599

nickpsecurity3• 11 years ago

That was a fun read. An insane solution I probably wouldn't have attempted. Impressed that they threw together a C++ parser and translator that worked that well in such a limited time.

kelvin0• 11 years ago

I could read stuff like this 24/7 ... something visceral about plowing though some ridiculous constraints and finally making history really gets to me ...

drzaiusapelord• 11 years ago

The PS1 had 2MB of RAM, and we had to do crazy things to get the game to fit.

That's what? About $80-100 worth of RAM back then? On a product that sold at $299 (July 1995 pricedrop) that's incredible. Nearly a third of your cost was ram alone.

derefr2• 11 years ago

Compare this to the Super Nintendo's 128KB of working memory.

It's hard to tell which games used more or less of that memory; the big thing about game complexity in that era was always ROM size limiting asset complexity, rather than RAM size limiting computational complexity, so the games released toward the end of the console's lifecycle were just ones with the biggest ROMs and therefore most assets, rather than games that used the available CPU+RAM more efficiently.[1]

Now I'm considering writing a memory profiler patch for a SNES emulator, to see how much of the 128KB is "hot" for any given game. I would bet the hardest-to-fit game would be something like SimCity or SimAnt or Populous.

On the other hand, the SNES also had "OAM" memory—effectively the 2D-sprite equivalent to GPU mesh handles. And those were a very conserved resource—I think there was space to have 128 sprites active in total? Developers definitely had problems fitting enough live sprites into a level. Super Mario World's naive answer was to basically do aggressive OAM garbage-collection, for example: any sprite scrolled far-enough offscreen ceases to exist, and must be spawned again by code. Later games got more clever about it, but it was always a worry in some form or another.

---

[1] There were also those that used expansion chips to effectively displace the SNES with something like an ARM SoC in the cartridge, once that became affordable. It's somewhat nonsensical to talk about how much SNES system memory some games used, because they came with their own.

ParrotyError• 11 years ago

Back in the days of the 8-bit micros there were some games that were absolutely incredible for what they could pack into a tiny machine. Elite on the BBC micro (32k) was one, but being poor I had a Spectrum 128. There was a helicopter simulator by Digital Integration for the 48k Spectrum called Tomahawk which was particularly good, and another 3D game called Starglider, which was originally for the Atari ST but there were ports to the Spectrum 128 and 48k Spectrum! I seem to remember an article in Sinclair User where they interviewed the developers who explained how they got it all to fit into such a small machine. Self-modifying code and using parts of the frame buffer to store code and data IIRC...

B0073D• 11 years ago

For those of you who don't want to endure Quora's forced account setup:

Here's a related anecdote from the late 1990s. I was one of the two programers (along with Andy Gavin) who wrote Crash Bandicoot for the PlayStation 1.

RAM was still a major issue even then. The PS1 had 2MB of RAM, and we had to do crazy things to get the game to fit. We had levels with over 10MB of data in them, and this had to be paged in and out dynamically, without any "hitches"—loading lags where the frame rate would drop below 30 Hz.

It mainly worked because Andy wrote an incredible paging system that would swap in and out 64K data pages as Crash traversed the level. This was a "full stack" tour de force, in that it ran the gamut from high-level memory management to opcode-level DMA coding. Andy even controlled the physical layout of bytes on the CD-ROM disk so that—even at 300KB/sec—the PS1 could load the data for each piece of a given level by the time Crash ended up there.

I wrote the packer tool that took the resources—sounds, art, lisp control code for critters, etc.—and packed them into 64K pages for Andy's system. (Incidentally, this problem—producing the ideal packing into fixed-sized pages of a set of arbitrarily-sized objects—is NP-complete, and therefore likely impossible to solve optimally in polynomial—i.e., reasonable—time.)

Some levels barely fit, and my packer used a variety of algorithms (first-fit, best-fit, etc.) to try to find the best packing, including a stochastic search akin to the gradient descent process used in Simulated annealing. Basically, I had a whole bunch of different packing strategies, and would try them all and use the best result.

The problem with using a random guided search like that, though, is that you never know if you're going to get the same result again. Some Crash levels fit into the maximum allowed number of pages (I think it was 21) only by virtue of the stochastic packer "getting lucky". This meant that once you had the level packed, you might change the code for a turtle and never be able to find a 21-page packing again. There were times when one of the artists would want to change something, and it would blow out the page count, and we'd have to change other stuff semi-randomly until the packer again found a packing that worked. Try explaining this to a crabby artist at 3 in the morning. :)

By far the best part in retrospect—and the worst part at the time—was getting the core C/assembly code to fit. We were literally days away from the drop-dead date for the "gold master"—our last chance to make the holiday season before we lost the entire year—and we were randomly permuting C code into semantically identical but syntactically different manifestations to get the compiler to produce code that was 200, 125, 50, then 8 bytes smaller. Permuting as in, "for (i=0; i < x; i++)"—what happens if we rewrite that as a while loop using a variable we already used above for something else? This was after we'd already exhausted the usual tricks of, e.g., stuffing data into the lower two bits of pointers (which only works because all addresses on the R3000 were 4-byte aligned).

Ultimately Crash fit into the PS1's memory with 4 bytes to spare. Yes, 4 bytes out of 2097152. Good times.

voltagex_• 11 years ago

Found while looking for other articles about this: http://web.stanford.edu/group/htgg/sts145papers/jdelahunt_20...

How Naughty Dog Fit Crash Bandicoot into 2MB of RAM on the PS1

Related Stories