Why We Threw out All Our Code (And Why You Should Too)

http://blog.nowjs.com/why-we-threw-out-all-our-code-and-why-you-sho

by sthatipamala • 15 years ago

92 64 15 years ago

blog.nowjs.com

nfm• 15 years ago

I've never used NowJS nor looked at the source for it, so I can't comment about their justification for a rewrite.

But most programmers I know don't need any encouragement to rewrite stuff, myself included!

boyter• 15 years ago

That's the attitude I am trying to avoid these days. When I look at something that I want to rewrite I now ask the following questions,

1. Do I have sufficient tests and testing to do this without breaking things?

2. Do I want to want to rewrite because I cant be bothered understanding it?

3. Is there going to be any gain now or in the near future by rewriting it?

4. Will I be able to rewrite it sometime in the future as part of another piece of work which actually delivers value?

Unless I come up with Yes, No, Yes, No I tend to leave it alone. There are other things of value I can do with my time.

EDIT - Formatting...

mvzink• 15 years ago

Thank you, these criteria just saved me from tearing down a building.

Someone• 15 years ago

It's managed to survive our vicious benchmarking tools and our ridiculously comprehensive test cases.

I guess they did not rewrite those from scratch. If so, I would guess they only rewrote, at the most, 25% of their code.

Volpe• 15 years ago

They threw out 3 months of work... That's not a rewrite, that's a prototype. People do this all the time.

josephcooney• 15 years ago

+100 - the biggest questions (to me) were HOW MUCH code did they throw out, and how old was it? Throwing out a few months of code you just finished writing (and which you're still intimately familliar with) is vastly different to throwing out a code-base that would be entering middle-school if it was a person.

neilk• 15 years ago

I don't know what the old codebase is like, but the current one is just 2145 LOC, not counting tests. 62K total.

I know that startups do way more that just write code, so I'm not criticizing them at all. And it seems they're not afflicted with the typical engineer modesty when it comes to their own codebase, which is a good thing. On the other hand it's not exactly a superhuman hacking achievement to rewrite this from scratch.

Swizec• 15 years ago

Rewrites are always fun and I applaud any man brave enough to undertake such an effort.

I've often been faced with a situation warranting a rewrite, but I never mustered the courage to actually go with a full rewrite I don't think. Gradual rewrites are much more my thing.

FuzzyDunlop• 15 years ago

I've been really really tempted to do one, but sadly the pressures of the job take precedence and so it usually ends up being the case that you stick with the code you've got and try and hack extra things in, knowing full well you really don't want to and aren't at all happy with it.

So my 'rewrites' sometimes come in the form of superficial organisation. Looks moderately nicer, doesn't work any better.

I guess it is courage though. You don't really know that your rewrite will be an actual improvement, or if you'll even pull it off. Brave move indeed.

raldi3• 15 years ago

Throwing out all your code and rewriting it from scratch is like demolishing your building and constructing a replacement from scratch. It's usually not the right thing to do, it occasionally and rarely is the right thing to do, and it's always the tempting thing to do.

It's stupid to argue, "You should never tear down a building" (like http://www.joelonsoftware.com/articles/fog0000000069.html implies) and it's also kinda stupid to write an article entitled, "Why We Tore Down Our Building (And Why You Should Too)"

Neither is universally right. The right thing to do in a given situation is a difficult and complicated question to answer, and it depends on the building in question: how big it is, how well it was designed, how well it's been maintained, etc.

praeclarum• 15 years ago

Right.

In this case it was 3 months worth of code. I think the analogy is closer to "we tore down this clay hut we built". Easy to build, easy to tear down.

If this were a mature code base, I would be a little more shocked.

Cushman2• 15 years ago

Or maybe closer to "We're putting this model building in the closet since we built the real one."

badclient5• 15 years ago

How often do you build a prototype building? Or a building as a side project?

Your analogy fails, sir.

It's easier and more common to write dirty code that functions as it should than build a building with poor foundation but that looks and functions as it should.

raldi3• 15 years ago

People build temporary structures all the time -- event tents, for example. And plenty of people finish a room, redo a bathroom, build a treehouse or a shed, etc, as a side project. And believe me, it takes a high level of expertise to inspect a building. Many of the most important kinds of flaw are only evident to the eye of an expert.

damoncali• 15 years ago

I think it's a good one. Think of commercial real estate. You have an old shopping center that is cash flowing. Its maintenance costs are high, and the rent is low. Once upon a time it made sense for the market and the style was perfect. Now it's out of date and the units are not exactly what people are looking for.

You could tear it down and build a fancy new one that would cash flow even more with higher rents and lower costs. It's exactly what the market needs today. Green energy, nice, open spaces, new paint. Do you break out the bulldozers?

It depends, of course.

jacobolus• 15 years ago

> How often do you build a prototype building? Or a building as a side project?

This is one of the most amazing things about rapidly growing cities in most of the world. Neighborhoods that start out as informal shanty-towns full of lean-tos with leaky roofs become, over the course of years, first neighborhoods with modest but permanent cement block buildings and tin roofs, and sprout all the typical urban amenities like power lines and sewers (often at first pirated ad-hoc from other nearby neighborhoods), have their roads paved, etc., and a few decades later look like normal neighborhoods, sometimes even surprisingly gentrified ones.

MartinCron• 15 years ago

My local zoo actually built a full-size prototype of their new penguin exhibit ahead of building the exhibit itself. Once people could actually walk around in the space, some improvements were obvious.

tkahn6• 15 years ago

Here's an example:

http://www.vt.edu/about/buildings/surge_space_building.html

"The building provides space to temporarily house academic or administrative units that are displaced because of renovations to their home buildings. The facility was designed using a pre-engineered steel frame with its exterior clad in metal panels and precast concrete-like panel accents. The building is designed to be disassembled and recycled in 10 or 15 years.

One floor; 45,000 square feet. Find a public access defibrillator in the hall intersect."

brudgers• 15 years ago

>"like demolishing your building and constructing a replacement from scratch. It's usually not the right thing to do" Actually, tearing down a building and rebuilding is very often the right thing to do, but isn't done for the same reasons people don't toss their code and rewrite from scratch - sunk costs.

Unlike software, new buildings are almost invariably significantly more efficient and less resource intensive than old ones - and green building has, of course, gone from a fringe movement in the wake of the Rio Summit, to the mainstream and is now standard for public and corporate buildings.

Like poorly written software old buildings are expensive to operate and maintain (and unlike well written software buildings still deteriorate over time when left only to their own devices). Finally, like software one must go through old buildings and figure how systems were designed and installed and patched in order to modify them - and this is very time consuming. Per square foot costs of renovation are commonly as much or more than new construction.

chegra• 15 years ago

Relevant: Things you should never do - http://www.joelonsoftware.com/articles/fog0000000069.html

benologist• 15 years ago

Throwing it away is easy when there's not that much of it.

My codebase is 10s of thousands of lines, it evolves as I get better but I'm not going to scrap it.

jjordan• 15 years ago

sigh; a headline like this is little more than HN linkbait. As others have mentioned, throwing out all your code is generally a bad idea, especially if your project is of significant size. How many lines is NowJS anyway? I doubt it was much of a loss.

dstein• 15 years ago

I'm still trying to figure out what NowJS does other than expose an RPC call using socket.io. All of the features on the website are actually socket.io features.

ctide3• 15 years ago

3. Sunk costs and egos get in the way of good software

Egos are what drive people to throw away their entire codebase and start over.

pyre• 15 years ago

Depends on the angle. The person that wrote the codebase is usually the one holding onto the old codebase due to ego. On the other hand, new people that come in and don't like things can be full of ego in the sense of, "I could program this better without any limbs!"

ctide3• 15 years ago

Egotistical engineers always think this time things will be different, because they're smart enough to do it right. The reality is that every rewrite brings a new and different set of problems that need to be addressed. You don't escape problems by scrapping it all and starting over and this is one of the most valuable lessons to learn as an engineer.

rickmb• 15 years ago

Egos and lack of skill.

It's so much easier to write from scratch than it is to refactor. So many programmers have no idea how to tackle an existing codebase, even if it's one they've wrote themselves.

ctide3• 15 years ago

Very important clarification, you are quite correct.

astrofinch• 15 years ago

It's probably easier to do this sort of rewrite if you've already got "ridiculously comprehensive test cases".

Anyway, congrats on 0.7 Sri!

d2• 15 years ago

Netscape 6 is (or was) the oft quoted example of why not to do this. All that old code may seem old and boring, but it has a huge amount of knowledge, wisdoms, learnings, fixed mistakes, performance improvements, great ideas, deleted bad ideas and security fixes built into it.

When you throw out your code you're throwing out all that knowledge on the assumption that you can build it all back with improvements.

Evolving an application from a known good state to a known better state is usually the best approach unless your codebase is small or your app has no traction.

Rewriting your app from the ground up is like getting divorced and dating a new girlfriend. The first few months are a lot of fun until you figure out how much you lost.

quink• 15 years ago

Without Netscape 6, we wouldn't have had Phoenix/Firebird/Firefox, and probably wouldn't have IE7, IE8, IE9 and KHTML/Webkit would have come about very differently and all these would have had much less competition.

Netscape 6, or at least the mere fact that it existed and worked well enough and the fact that it was the first browser ever that did standards well and focused on not much else other than standards, made it the most important browser ever created.

Netscape 6 was the most revolutionary browser project ever created and had the development effort behind Netscape 6 not been done, we would possibly have had a much worse very IE-only web today.

Never mind that beyond all that, Netscape 4 was terrible and deserved to be thrown out.

shadowfox• 15 years ago

> Netscape 6 was the most revolutionary browser project ever created

I dont disagree with the general sentiment of your comment. But as someone who attempted to use Netscape 6 at various stages of its development, I fear that statement might be stretching facts a little bit.

steve8918• 15 years ago

Once you start making revenues from your code, a complete rewriting is probably the worst thing you can do for your company. Sometimes it might be warranted, but customers care about solving problems, not about technology.

Chances are if you wrote properly modularized code, you can rewrite modules one by one, which is perfectly legit. But just dropping everything and trying to rewrite a product from scratch is a mistake that a company usually only makes once (before they die).

raganwald• 15 years ago

A few people have pointed out that it’s easy to throw out your code when you (a) have tests for it, and (b) only have three months invested in it. I agree with both of those points, but want to tip my hat to a team that rewrites their code after three months.

The reason some code bases can’t be rewritten after a few years is because when the code was only three months old, the team passed on the chance to rewrite it while it was still a tractable exercise.

grncdr• 15 years ago

Very much agree. An extra layer on the danger cake (for web apps) is if your user base grows slowly while keeping the business profitable. Suddenly (sic) you hit real capacity limits after n years, and have no idea how the "organically grown" code in question works.

kellishaver• 15 years ago

I'm glad they're happy with the rewrite, but honestly, this makes me feel a bit nervous about updating my Node app utilizing NowJS to the 0.7 release.

Maybe it is better. Maybe it is more stable, smaller, and faster, and well-tested. But it hasn't had as much "real world" use thrown at it as the previous codebase had.

I'm not saying I won't upgrade, but I'll likely wait a bit until I do and stay with what I know works right now, at least in production.

brandonwang• 15 years ago

Please bear in mind the counter article everyone is citing (http://www.joelonsoftware.com/articles/fog0000000069.html) was written 11 years ago. I'm not saying that it's wrong because it's from 11 years ago, but you really want to consider its central example.

Netscape 6.0 was a complete rewrite. The article claims that this was a mistake.

Now, if you would recall, one of the reasons Netscape 6.0 was completely rewritten was to use Raptor/NGLayout. Raptor/NGLayout was later renamed to Gecko. Gecko is the currently developed Firefox rendering engine, and Netscape 6.0 marked the start of the Mozilla foundation.

I would harshly argue that Gecko would not have taken off nearly as well without Netscape's rewrite. But this is definitely not a story about Gecko's popularity. It's about the reason why Gecko was the decision they took at the time, and the current impact of Gecko.

Besides the fact that Gecko was one of the early open-source success stories, the reason why Netscape decided to scrap everything and start from scratch was simple: Netscape 4.0 was horrid. Remember back to the days of IE vs Netscape. Netscape was significantly slower, and had no dynamic HTML among many other features. It was buggy and would crash frequently. And the development on the "continuation" of 4.0 code was simply too slow to compete, because of outdated, not modulated code, and stuff stuck in that simply "worked," and never changed to allow for future changes.

Now, people attribute Netscape 6 to Netscape's death because of this radical decision. But take a look at Netscape's usage statistics: http://upload.wikimedia.org/wikipedia/en/1/16/Netscape_Navig... . Hey, guess when Netscape 4.0 came out and IE started to dominate? 1996-1997. By 2000, when Netscape 6 was released, Netscape's usage already dwindled down to nothing.

Now, you're thinking, why didn't Gecko save Netscape? It takes too long to reverse these tides. Netscape was already dead. AOL acquired Netscape in 1998, to combat IE. Then, in 2003, AOL won an antitrust suit against Microsoft, and allowed AOL to use IE royalty-free. Netscape was pretty much scrapped at that moment because the damage had been done.

But wait.

Mozilla released Phoenix in 2004, based on Gecko. You should now the story from there. (Clue: Phoenix -> Firefox)

Disclaimer: I'm not saying that Firefox > IE or Chrome or whatever. I'm saying that the codebase Netscape 4.0 was based on was doomed to fail. Gecko still lives on today, and was started as a rewrite of Netscape.

Anyway, my point is, there are times to rewrite things, and there are times to keep them the same. If you have a good, legitimate reason to rewrite, as Netscape or NowJS did, you should. If you are just the stubborn programmer that always implements his own Queue or QuickSort when libraries exist, don't.

dasher• 15 years ago

Holding onto code just because is a foolish notion. Sometimes approaching things with a fresh perspective is what's best - it allows you to apply what you've learned so far and actually improve not just one section of code, but the whole kit & caboodle.

This is one of the reasons you should write tests as you develop - it ensures you have a consistent baseline you can apply.

As a developer you are constantly learning & evolving. If you're not confident enough to draw a line under what you've done, step back and evaluate your position and rewrite it when needed - that's fine, but others find it an effective approach to building something bigger, better and more sustainable.

tedkalaw• 15 years ago

I think that this applies to the technology you're using as well.

I've been thinking about this because I'm writing an application in Rails and realized relatively recently that it would be better suited as a Node application. My motivation for using Rails was that it was what I knew the best. Luckily for me, the most time-consuming part was the front-end, not the Rails part, so I'm hoping my switch to Node will be relatively painless.

jerf• 15 years ago

Throwing away Netscape 4 and rewriting it from scratch wasn't a bad decision because nothing good ever came of it; it was a bad decision because it killed the company. The shattered remnant of Netscape that hobbled across the finish line to release "Netscape 6" wasn't even Netscape anymore, it was a rapidly fading division of AOL, IIRC, that just stamped its name on a Mozilla product it had little to do with producing at that point, to widespread derision.

As bad as Netscape 4/5 may have been, while I can't guarantee trying to refactor instead of rewrite would have gone better, I can guarantee it couldn't have had any worse a result than the Total Death that was the actual result.

And the phoenix-return of Mozilla is the exception, not the rule.

fpgeek• 15 years ago

To be pedantic, there is at least one fate worse than the Total Death we observed: Total Death + no Gecko.

Even though ths external case is stronger, this holds true even from the company's perspective. AOL continued to release Gecko-derived browsers through 2007. They got something from that (even if it was just AIM installs and propping up the fading Netscape brand for other purposes).

alttag• 15 years ago

In support of your comment, I think some are forgetting how horrible Netscape 6 actually was to use. (Or am I just getting old? It wasn't that long ago!)

Compared to earlier versions it was incredibly slow—even by the standards of the time—and crashed a bit more frequently. It took much longer to load or perform some common tasks. Sure, the rendering engine was better for many pages, but browser sniffers were still common, and (like now) weren't updated often enough, so bad versions of sites got served occasionally too.

There were a host of reasons why Netscape 6 was bad from a user's perspective. Spolsky has it right when he says Netscape 6 was a piece of junk. Let's not allow time passing to permit us to remember it more fondly.

Sure, future versions got better, but that was due to he creation of a new "corporate" memory. It took a while to get over having discarded so much learning.

beck5• 15 years ago

An attention grabbing headline but all the same I have been using now.js and its been a great tool (although a cut down on the print statements would be nice).

makecheck2• 15 years ago

Justified or not, I think a project owes it to its users to change major version numbers when a major rewrite (much less a complete one) has occurred.

As perfect as your code might be, customers deserve a hint when major changes were done under the hood. Moving from 0.6 to 0.7 screams "just a few changes, some new features", not "we threw everything out and started over". This should have been made into "1.0beta", I would think.

stephth• 15 years ago

There are other factors involved when deciding a version number than the code under the hood: API changes, new features, is it production ready, 'this is the now stable API that we'll be maintaining for a long time', etc.

Edit: for perspective, a thoughtful proposition on how to approach software versioning: http://semver.org

makecheck2• 15 years ago

I agree that there are many factors. I'm just saying that some changes are important enough to trump anything else; the presence of a rewrite should decide the version all by itself.

anirudh• 15 years ago

Its not always the right thing as there are years of issues/corner cases that have been resolved.

flocial• 15 years ago

Well, just because some prominent luminary says it shouldn't be done never means it's always a bad idea. A lot of the basis against rewrites are for relatively mature projects still on a high growth trajectory or large installation base. However, there are exceptions to that as well. Foursquare went from LAMP to Scala/Lift, PostgreSQL, and MongoDB with minimal complications and it's paid off handsomely to support their amazing growth.

Also, the money quote: "It's managed to survive our vicious benchmarking tools and our ridiculously comprehensive test cases. " My assumption is they are using the same languages and tool sets (not porting from language X to Y).

I'm curious as to how long the rewrite took though.

stretchwithme• 15 years ago

If you're dealing with human DNA, a complete code rewrite will likely fail.

But if its just an amoeba and you have a lot of time, go for it.

shahan• 15 years ago

Whether their decision was right or wrong, they made a decision that big companies tend not to make. Startups should probably have a bias towards decisions like that.

jheriko• 15 years ago

I'd be dubious that they actually rewrote everything - having working stuff lying around that already worked, some of which was probably just fine, I'd hope it got copy-pasted into the new code...

adrianscott• 15 years ago

You should read the book, "In Search of Stupidity - Over 20 Years of High-Tech Marketing Disasters"

Refactoring is almost always preferable to throwing out all your code, imho.

kayoone• 15 years ago

Especially when getting into a codebase that is not yours you are tempted to think "why dont i just rewrite this in a better an cleaner way". Then you go on todo that and fall into most of the pitfalls the original author fell in before, have to rethink every idea behind and it usually is alot more work than you think. That said it depends heavily on the size of the project and a bunch of other things, for a 3 month old project it might make sense.

onan_barbarian• 15 years ago

Frequently, throwing out "all your code" is a sign you did pretty much everything wrong to begin with. Personally, the sign of a decent design is when you can wind up doing the old "Grandpa's Mattock" trick:

"This is grandpa's mattock; my father replaced the handle and I replaced the blade".

nvictor• 15 years ago

Be ready to throw away your first version. I wish I had the right quote on hands. The K in K&R said that.

Why We Threw out All Our Code (And Why You Should Too)

Related Stories