A remaster with no old code: Crash Bandicoot was rebuilt nearly from scratch

https://arstechnica.com/gaming/2017/06/a-remaster-with-no-old-code-crash-bandicoot-was-rebuilt-nearly-from-scratch/

by Tomte • 8 years ago

131 76 8 years ago

arstechnica.com

motwera3• 8 years ago

For those interested about the mesh level geometry, you can give the SDCC 2017 panel a watch.

https://www.youtube.com/watch?v=UZAetHANJJ4

https://www.engadget.com/2017/07/03/crash-bandicoot-n-sane-t...

klodolph• 8 years ago

This is buried in a comment but it's worth mentioning.

Naughty Dog used Lisp to write Crash Bandicoot. They had something they called GOAL--Game Oriented Assembly Lisp. From what I understand, this was a DSL they wrote so they could write assembly language using Lisp syntax, and call it from Lisp functions.

https://web.archive.org/web/20070127022728/http://lists.midn...

KwanEsq• 8 years ago

GOAL was the next iteration of that idea, used in the Jak games. Crash used GOOL, which you can read more about at https://all-things-andy-gavin.com/2011/03/12/making-crash-ba...

bitwize• 8 years ago

"Any sufficiently complicated Lisp or Scheme program will eventually be rewritten in C++, Java, or Python."

It's as true now as when I first said it in 2005-2006.

sova• 8 years ago

Don't you mean that the other way around? "Any C++, Java, or Python, or <insert your lang here> program will eventually be written in a Lisp-based DSL that generates that language" ? Seems far more accurate to me than to claim that all LISP functionality is derivable in (more rigid, less flexible) language. What prompted you to say this in the first place? If LISP is the mother language, of course all the rest are but a subset.

drewblaisdell• 8 years ago

Isn't HN itself a counterexample?

nicoburns• 8 years ago

Obligatory link to this series on the making of the original (written by one of the developers):

https://all-things-andy-gavin.com/2011/02/02/making-crash-ba...

mmastrac• 8 years ago

This was insanely interesting. Thanks for the link.

corysama• 8 years ago

The only method I’ve seen work semi-reliably for archiving console game source is to grab the physical machine that made the final build, spray paint “Do Not Dispose! Build archive!” on its side and stick it in a back room. Even that has a 2% chance/year to end up in a dump.

Otherwise, console game build systems were far to finicky to repro later (at least up to the 360/PS3 generation). You would need a huge collection of confidential tools from the manufacturer and those tools sometimes had OS/driver/hardware dependencies that can be reproed reliably with a VM. Think USB driver for the GameCube debugging connection. IIRC, the PS1 and N64 required add-in cards to connect a PC to them. Then there was the propriety test disc/cartridge burners.

le-mark• 8 years ago

All true, yet for PS1 there was the net yaroze dev system that was distributed for some time. I actually had a copy at one point.

Seems inconcievable it wouldn't just been easier to use a software emulator and the original data from the cd. But then I don't know if there is a good software emu for the PS1?

https://en.wikipedia.org/wiki/Net_Yaroze

motwera3• 8 years ago

It's likely that Vicarious Visions was unable to get the source code from Naughty Dog due to Activision being unable to get it via Sony, though I'm not 100% sure about that since VV mentioned that majority of the original code is virtually unusable in a blog post.

https://blog.us.playstation.com/2016/12/03/crash-bandicoot-n...

kingbirdy• 8 years ago

Interesting, wish it had more details though. Also, the title should include (2017)

danShumway10• 8 years ago

The games industry is so stinking bad at archival that it hurts. Very little consideration is paid to using architectures and build tools that can be replicated later. Source code becomes unavailable (as was the case here). Game logic is written using highly platform-specific code that's difficult to emulate. Source textures and other raw resources get thrown out.

And not to diminish from the accomplishment of the team here, because it must have been a huge amount of work, but even with that amount of work this recreation ended up having problems with collision detection that changed the entire feel of the game. I don't think that's because the devs are bad, but just because the task they were trying to undertake was so unbelievably impossible -- of course they missed stuff.

If you're in the games industry, please put some effort into preserving your work for history. It does actually matter.

GatoVolador• 8 years ago

Paid, not payed.

JasonFruit• 8 years ago

Archival is an adjective. I will die on this hill.

user3359• 8 years ago

IIRC from developer stories from this era, things had to be done "creatively" to save space, and ensure optimal runtime execution. Certain platforms may well have had undocumented bugs and platform-specific optimizations that you simply had to adapt to.

Archiving wasn't a consideration - launching a playable game that would pay the salaries and lead to future projects was.

We take for granted that websites but in 95 are still more or less viewable in their original format today, but that's because things are designed that way. Gaming platforms were more or less abandoned entirely and rewritten - I might be wrong here but games from PlayStation 1 are not playable on PlayStation 4 are they?

danShumway10• 8 years ago

Yes and no. I don't really have a justification for saying this because I was never working in that environment, but I don't think the industry spent enough time thinking about how their games would be played in the future and I suspect that a lot of the "well, the platforms required us to do this" is an excuse.

Yes, there are platform-specific quirks and bugs and stuff to work around, but say you're taking advantage of a console specific hack -- well, is that hack encapsulated? Is that hack mentioned anywhere in your internal documentation? Do you have internal documentation? I think you're mostly right that the thought was, "why would we care about building a game that runs on PS2, we'll just make a new game for that console."

You can do a lot with even really bad source code if the other stuff is good -- if you still have uncompressed assets, or your build process isn't completely reliant on some weird third-party proprietary dev kit. Bear in mind that with this project, they didn't even have access to the source code, and they had to manually decode 3D mesh data. So platform-specific logic was the least of their problems.

I suspect that part of that is that the games industry is just really young. And maybe some people were yelling about archival back then, but there weren't any real tangible consequences. Now that remasters are a big thing, you've got situations like this, where companies want to leverage old IPs and are suddenly realizing that doing that is twice as complicated as it should be.

agumonkey• 8 years ago

It's a fair point. See emulated snes games that fail on some vfx because code relied on ultra specific hardware architecture.

techopoly2• 8 years ago

Correct, PS1 games are not playable on PS4.

reificator• 8 years ago

even with that amount of work this recreation ended up having problems with collision detection that changed the entire feel of the game.

You know, I did 100% minus (non-required) relics on all three games, and I didn't really feel like it was any different. Sure I didn't compare them side-by-side, and sure I did explicitly ignore the speedrunning part of the games where minor differences would be easier to notice.

But it felt like the same games to me...

danShumway10• 8 years ago

https://www.eurogamer.net/articles/2017-07-04-fans-may-know-...

I'm curious if they ever patched this. It's certainly subtle enough that it got past their testing, so I'll give you that "changed the entire feel of the game" is probably an over-exaggeration.

monocasa4• 8 years ago

Crash Bandicoot was written mainly in GOOL, a custom to Naughty Dog lisp variant.

The game would have had to be rewritten anyway almost certainly.

danShumway10• 8 years ago

I would think the lesson from this is, "don't build an in-house custom Lisp variant for all of your games unless you're willing to distribute the documentation, compiler, and dev tools with your source code."

zeta0134• 8 years ago

I'm not convinced it'd be impossible to use the GOOL code though. It should in theory have been plausible to port a GOOL interpreter, and then simply use the original code in the new interpreter. That said, there are some possible complications with the approach, depending on how tightly integrated the old GOOL system was with the rest of Naughty Dog's very custom engine.

Of course they'd need to actually have the GOOL code to start with. It sounds like all they got from the original games was the original meshes and some other non-code assets, which ... I mean, that's some incredible reverse engineering work. I'm much, much more forgiving of the tiny little inconsistencies here and there given that context; the team did a fantastic job if they were primarily going off reference footage.

dmbaggett3• 8 years ago

Not really. The GOOL code was used for character logic and things like the load/save screen. The majority of the code was written in C and MIPS assembly.

jmkni2• 8 years ago

Fun fact, when Metal Gear Solid (1) was ported to PC, the work was done by a handful of people, and they didn't have access to the original resources so they found some of them on fan websites. Some of the audio files used in the release are 30 second loops of much longer pieces of music from the original game.

monocasa4• 8 years ago

Huh, that's weird. PS1 assets were remarkably transparent. Generally the audio was just CD-A tracks or ADPCM files, and the rest of the data was a ISO-9660 filesystem that showed up when you stuck the disc in a computer.

jlg232• 8 years ago

Let me hand you some binoculars, it might be hard to see the details from the vantage point of giants' shoulders:

* A lot of things "missed" is what further research proved to be "obvious" conclusions of exactly the groundwork laid by these games.

* Between 80h/week, personal life and the permanent question whether your game will sell at all, considerations what you should do in case your game becomes a cultural heritage are moot.

I have been part of this circus 15 years ago and while I definitely am proud of what I did, I do not think any lessons I learned are directly relevant today. Not only that, but I can not imagine to revisit my old code for any amount of money. Yes, my build system increased build speed by a factor of 200 over industry standards, but no, I am not going through those 2.5k loc gnu-make ever again. Or to even document the tricks we used to deliver a 80k jar, including assets.

A lot of the stuff was done to deal with hardware limitations that are irrelevant today. Why go through a lot of pain when the resources today are sufficient to play 100 instances of an unoptimized rewrite in an emulator?

danShumway10• 8 years ago

So I'll say up front, you're right to say I don't have 15 years of experience to back this up. And if your takeaway is, "this idiot doesn't know what they're talking about", fine, I'll take that. That's a completely reasonable reaction.

But, you're wrong.

The top rated post on Hackernews right now is a bunch of people gushing about how cool it is to see the source code for the Apple II LOGO language[0]. Your code has value beyond whether or not someone is going to pick up a new game-dev trick for their next title. It's a window into how games used to be built. It is a tiny composite piece of gaming history, and history is not just about how Mario was built, it's also about how small games were built, and what trends across the entire industry looked like.

Your code has value beyond just publicly releasing the source. Maybe you would never revisit it in a million years for a million dollars, but the overall gaming industry is interested in remakes and remasters right now, and if you've ever worked for a publisher or sold the IP rights to one of your titles, it is not out of the question that the publisher might want to do a remake without you. That's the entire premise of this article, and if you have the source code, even if it's not portable, even if it's using hardware-specific hacks, even if it's written in an obscure language, your publisher will be in a better position than if you didn't.

To be kind of blunt, this attitude is exactly what I was getting at above. Game devs don't understand why they should even care about history. If your takeaway is, "I don't care about old games unless they make my new PS4 game run faster" you have missed the point. If your takeaway is, "archival only matters if my game is successful, and I don't know if it'll be successful yet", you have missed the point.

I completely, 100% understand that game dev is stressful. Zoom out a bit from that; working in the games industry in general is kind of hellish right now. If you're putting 80 hours a week into building a game for (on average) less pay than you'd get as a general software dev, and you're not doing it because you genuinely care about what you're making... that's a stupid career choice. And if you do genuinely care about what you're making, then try to make sure it lives longer than you do.

Bear in mind, what I'm asking for doesn't have to take much work. You don't have to go crazy building out your source-code in a special way if you don't have time. Change nothing at all about your dev process -- just take 3 minutes when your game ships, stick your raw source code, assets, and a DRM free binary on a flash drive, and then mail it to Jason Scott.

That's it. Just by doing that, you're now ahead of most other companies in the games industry right now.

[0] https://news.ycombinator.com/item?id=18142287

tiredwired• 8 years ago

The lack of archiving is due to the frequent layoffs at the end of projects at game companies.

faitswulff• 8 years ago

First paragraph of the top comment on Ars:

I worked at Insomniac on Spyro while Naughty Dog was working on the Crash series. We had a friendly competition squeezing out the maximum possible performance from the PlayStation, and that turned into an even friendlier coopetition on our respective PS2 engines when we actually started sharing some code. Based on what I saw of both code bases, Naughty Dog did them a huge favor by sharing so little. It would have been more work to rehab than recreate.

Highly recommend reading the rest of the comment, too!

Rebelgecko• 8 years ago

Assuming it wasn't a typo, I really like the term "coopetition"

toyg• 8 years ago

It is a problem that game development shares with other forms of mass-marketed art: film industry, tv, music... they are all pretty bad at conservation, and were downright atrocious until very recently (I’m sure you heard how the BBC lost the first few seasons of Dr Who — and they are an excellence for conservation in the field).

Simply put, the emphasis in creative industries is to continuously create (eh) new stuff, not revisiting the old. Couple that with inevitable financial pressures, add the cultural distortion of software development culture at large (where everything changes every year or so, and if you are not on the latest and greatest you are dead in the water) and you have a situation where conservation is the very last priority.

CJefferson• 8 years ago

The systems we run on make it very hard. With emulation, the best way to run an old game is a final binary.

I wrote an iOS 3 game. I have a lovely git repository of code. Is fairly useless as apple won't give me either the OS or compiler I need to run the code.

acoye• 8 years ago

They rebuilt a game in Unreal 4. So yeah from the ps1 era almost everything had to be redone from scratch. I fail to see the problem here. Did player expected them to package the optimized lisp virtual machine they had back then in UE4?

gmueckl3• 8 years ago

Reusing prtions of the gameplay code in a remaster can be feasible. Other remastering projects have managed to go further, although they had more favorable starting conditions. The remastered Full Throttle game is running the original SCUMM game logic without any changes. They instead extended the VM to perform on the fly asset swaps for the remastered artwork and sound.

jplayer01• 8 years ago

Did player expected them to package the optimized lisp virtual machine they had back then

Well, yeah? It's a remaster, not a remake. A remaster implies taking an existing project and updating it. And it makes sense to keep the old game logic - why write from scratch if you already have the whole game in a working state?

motwera3• 8 years ago

It's a remake, not a remaster in the definition that you are thinking of. Vicarious Visions did not use any of the code, likely to the fact that either it was unusable or they did not ever have the source code.

N. Sane was done in Alchemy Engine, not Unreal Engine 4 (unlike Reignited), although VV is likely following suit with their upcoming project.

sotojuan• 8 years ago

So was Kingdom Hearts. FFVI Steam and PS4 versions had to be made from the original PC version by Eidos since the PSOne original's source is lost.

efrafa2• 8 years ago

Funny story about Crash: Everybody around 30 years old in Slovakia knows Crash Bandicoot. In late 90s - early 00s there was a very popular TV show where you could call in and play the game with your phone (basically just left/right/jump) I spent hours watching it https://youtu.be/v3RYappcWZg

Waterluvian• 8 years ago

Oh my. And they show a bunch of toy prizes. Did this cost money to call?

efrafa2• 8 years ago

I think it was like regular domestic call and you could won playstation at the end of the month. But since it was super popular, it was crazy hard to get through, I tried, but lines was always busy :)

chamakits• 8 years ago

Oh wow this is cool.

Is this real?

Like even though the players die a good deal, it still feels like they can play it pretty smoothly.

I’d love to know the system that got built out for this. I imagine it was detecting tones, but would love any more insight into all of that. An rpg game would have been a good way to overcome any timing issues games may introduce.

acct1771• 8 years ago

Yeah, but then it's not exciting.

moepstar• 8 years ago

There was a similar show, called "Hugo" in Germany in the same timeframe...

Even with accompanying video games - ah, the memories...

https://en.wikipedia.org/wiki/Hugo_(game_show)

dmbaggett3• 8 years ago

That is so awesome! Thanks for making my day. :)

A remaster with no old code: Crash Bandicoot was rebuilt nearly from scratch

Related Stories