Lessons learned from rewriting code

http://huseyinpolatyuruk.com/2019/02/04/lessons-learned-from-rewriting-code-in-my-10-years-as-a-developer/

by hpolatyuruk • 7 years ago

116 131 7 years ago

our CTO was handling everything about AntiMalware. He was the main developer...

There's (one) of your problems. Your CTO (just like managers) should not be coding, they should be defining and leading the technical strategy of the company.

reallydude• 7 years ago

they should be defining and leading the technical strategy of the company.

At most companies (small to medium and a large number of industries), that isn't a fulltime job. Most companies have IT as support or are a very small segment of the total business.

wolco• 7 years ago

If you happen to be part of a two person team without any money you're CTO is going to need to code. At a team of 4/5 the handoff begins by employee 11 they shouldn't be coding perhaps firefighting if necessary.

jasonpeacock3• 7 years ago

As part of a two person team, there are no CEO/COO/CTO/CFO roles...you're just two people building something.

It's a joke to give yourself a management title when there's nothing to manage.

usrusr• 7 years ago

In a two person company you don't have a CTO and a CEO, you have "The Code Guy/Gal" and "The Other One".

cosmodisk4• 7 years ago

All these CFO,CTO,CEO,CMO and etc. titles make me laugh when it comes to small companies. These titles make sense when we talk about the likes of Google,MS, Facebook. If it's a shop of 10 people, you are not a CTO,you are a tech lead at best with a few less experienced developers.The 2 co-owners of the business I work for call themselves joint CEOs, with only one line of management separating them from the lowest level employees.I could call myself Vice President or COO or something like this but that'd be idiotic knowing that I only manage a team of 10 people.This obsession and vanity with titles is complete nonsense.

Numberwang• 7 years ago

This makes me think of groovehq who just recently forced all their customers on to their new rewritten from scratch product.

This after one full year of their customers begging them not to.

So now they have a new modern (in the 2017 sense) product which is worse in many ways and with a whole set of new bugs. And some very unhappy customers.

Improve your products. Don't replace them.

kostarelo• 7 years ago

Wow, I kinda work in a legacy (3-4 years old), not very easy codebase, where new releases affect other parts and bring new bugs, but I cannot imagine the state of yours. Not being able to do an update?

joefourier3• 7 years ago

How can a 3-4 year old codebase be legacy? That's not a lot of time for the platform or technologies it uses to become crusty and obsolete.

defined• 7 years ago

Some codebases are legacy from the word go. Legacy is more about the maintainability than the age of the codebase.

randallsquared2• 7 years ago

Maybe it wasn't built with containerization in mind. ;)

bdcravens3• 7 years ago

I often surprise myself - I see commits that are 4 years old that I could have sworn I implemented just a few months ago

notdang• 7 years ago

maybe it's javascript

davidjnelson2• 7 years ago

In the book Working Effectively With Legacy Code the author defines “legacy” as “code without tests”.

c-smile2• 7 years ago

I am working on Sciter (https://sciter.com) that appeared in Norton Antivirus in 2007, so it is used in production 12 years already.

And I never did full rewrite. The development started as "C with classes" when std:: was not the thing so I had and still have my own array<T>, string<T>, hash_map<T>, etc.

Since then I've added GPU support, Mac and Linux versions, scripting … a lot of features. But not full rewrite and no need for that that far.

Only incremental refactorings when they needed, e.g. when all platforms got reliable C++11 compilers I started using std::function instead of those ugly inner structures.

Am I lucky or what?

novok2• 7 years ago

Why do you think electron is more popular than sciter? They seem to target similar things? Is 'sciter html' a bit different than 'chrome html', so you can't make a web app out of the same code base too?

bdavis__3• 7 years ago

the usual definition of "legacy system" is a system that uses tools, technologies, and techniques that an organization would not use for new development. so, in SF, things change on a time scale of weeks. your 'go microservice' might be considered a legacy system. here in fly over country, COBOL on the 360 might be considered apropro for use today.

devxpy• 7 years ago

I never really got the reason for doing this

    var condn = ...;
    if (condn) {
        return true;
    }
    return false;

bberrry4• 7 years ago

Because it gives you the opportunity to descriptively name the expression, sparing others from parsing it mentally.

Liron4• 7 years ago

Do you also think `if ((x > 3) == true)` is more descriptive than `if (x > 3)`?

sombremesa• 7 years ago

Can't the returning function be named appropriately?

pdkl95• 7 years ago

Storing the result of a Boolean expression isn't the strange part. Storing it in a well-named variable can make the code a lot easier to read; the name (hopefully) communicates intent, and it can separate a conditional action into separate steps.

The strange part is testing Boolean value... to return the same value as Boolean constants.

Why wasn't the Boolean result simply returned directly?

    var condn = ...;
    return condn;

The if statement isn't necessary. Even if you wanted to explicitly return only "true" or "false" - using the if statement to convert truthy and falsy results into true booleans - using double not operators removes q potentially mispredicted branch.

    var condn = ...;
    return !!condn;

tuyiown• 7 years ago

It's been discussed here before, I don't buy the descriptive argument per se, it's the function name purpose, but another argument is to anticipate further processing then condition description can become purposeful, and you just have to add code in the right branch

TuringNYC2• 7 years ago

Perhaps you need to log the value of “condition” twice (once in each of 2+ cases) and you don’t wish to evaluate condition twice.

mjw1007• 7 years ago

One reason is that a code-coverage checker will tell you whether you're testing both cases.

roel_v• 7 years ago

Maybe there once was a breakpoint at the return false, or an assert, and it was just left like this in case the breakpoint is needed again.

indentit• 7 years ago

Modern debuggers generally support conditional breakpoints, but they often incur a high performance penalty, so this reasoning makes sense to me.

Liron4• 7 years ago

He lost me at the very beginning due to clumsy handling of boolean values.

Liron4• 7 years ago

Joel Spolsky wrote a classic piece about Netscape's rewrite in 2000 [1], the article links to it too.

[1] https://www.joelonsoftware.com/2000/04/06/things-you-should-...

FascistDonut• 7 years ago

This is referenced in the article.

kasperni• 7 years ago

"Rewriting a system from the ground up is essentially an admission of failure as a designer. It is making the statement, “We failed to design a maintainable system and so must start over.” "

Or maybe the domain you are working with was vastly more complex than you imagined, so your nice little system turned out to be more of a prototype than a finished product. Or maybe the features needed was vastly different from what you expected when you started implementing the thing. I'm sure everyone can come up with a million other reasons.

But the problem I have every time with these "Never rewrite your software" stories. Is that they are almost always one-sided. You rarely hear about the stories where a company decided not to do a rewrite and was out-innovated because every little feature took months to implement. Or failed because they could not attract competent developers because their codebase was a complete mess.

Yes, the rewrite might have been a partial or complete failure. But would the patient have survived without the operation anyway?

nradov2• 7 years ago

Or maybe your customer base and system load has expanded by an order of magnitude beyond the original design.

save_ferris• 7 years ago

Writing scalable software is all about being as flexible as possible. If system load expanded by an order of magnitude and the original design wasn't flexible enough to scale with the load, that's a bad software design.

goto114• 7 years ago

This should not require a ground-up rewrite though, unless the code is a big ball of mud in the first place. You might need to change the overall architecture (e.g split monolithic code into to multiple services) and/or optimize some critical bottlenecks, but most code (business logic, UI etc.) will be unaffected.

michaelvoz• 7 years ago

Rewriting is a very healthy part of software engineering. Sometimes you just didn’t anticipate what the company or product was going to do. The solution you wrote for 1k/qps probably won’t work at a million qps, and it is kind of silly to expect it to.

bdcravens3• 7 years ago

Sometimes when you're designing a system, you acknowledge it's bad (ie, technical debt). Ultimately, your purpose is supporting the system (typically the business), not crafting artisanal design and code. I suspect there are many beautiful git repos that are deleted when companies close their doors.

randallsquared2• 7 years ago

I'm sure everyone can come up with a million other reasons.

Indeed. One reason for rewrite is that there isn't a smooth continuum of changes from vCurrent to vFuture. If the discontinuity will have to be bridged anyway, and if other intermediate changes aren't going to bring sufficient improvement, jumping into it might be the most efficient way.

gfodor2• 7 years ago

In finance there is the concept of "Risk Adjusted Returns" -- in the case of a re-write, time efficiency is not a sufficient conceptual framework for talking about it, because it's risk laden. For example, I would much rather take 2x as long to accomplish the goal, if its 10x less likely to result in catastrophic failure. A failure to recognize the risk vs time tradeoff I think can lead to a failure of being motivated enough to "work the problem" -- often there is a continuum of changes that exists -- and it might not be "smooth" and could be in fact quite painful -- but it will be way, way less risky. It's often so less risky to transition software incrementally vs bet the farm on a rewrite, I feel like in most contexts a rewrite should just be ruled out completely without further discussion: better to just muddle through and try to figure out a way to survive without it, since you may come up with a better solution that is less risk laden just by waiting (due to new ideas, new technology, new market opportunities, etc) and not stalling your current product development and risking that you'd effectively destroy everything of value you've created so far due to total stagnation.

crimsonalucard5• 7 years ago

I would say 90% of the time, there is a failure in design.

Most of the time the designer does not fully understand the problem because the product hasn't been tested yet. Additionally even after the product has been rewritten 90% of the time the new design is wrong.

Often the second design is slightly better or worse than the first design because the second design is also a shot in the dark. Why?

This is because there is no way to theoretically prove a design is the correct solution to the problem. The only way to prove a design is to test it. There is no science or proof behind "design" only "design principles" and anecdotes... all of which can be broken and tend to evolve over time.

This is why designers tend to be artists rather than scientists. Designers are hired for problem spaces where no theoretical solution exist.

If most designs are wrong how do products even exist? Well computers are fast enough to cover up design flaws. Hacks can be made to bridge gaps in designs. Overall designs are wrong but can be patched to make anything work.

yeahitslikethat• 7 years ago

Or, did a rewrite and it launched them to success.

hinkley• 7 years ago

That doesn't sound like an 'or' to me. That sounds like a 'why'.

A rewrite also means no new features for a very long time. And my very cynical view is that people asking for a rewrite are motivated by the peace and quiet that happens during the first six months of the rewrite. In a world where people leave after 3 years and get a pass for their first few months that represents a pretty large fraction of their tenure.

But it's a bit like rehab. You've decided to start over with a new set of habits and hope it sticks. But like rehab or new year's resolutions, the relapse rate is high.

My current take is that there is a paradox here. The teams that deserve a rewrite usually don't need a rewrite. There's a high degree of overlap between being capable of fixing your broken crap properly and being able to refactor to a better system.

The only rewrites I've seen work look like the Ship of Theseus. As far as management is concerned it's not a new system. But the old developers can barely recognize the code.

GoToRO2• 7 years ago

I can not comment unless I see the code(s).

ChristianGeek• 7 years ago

And yet you did.

GoToRO2• 7 years ago

Did I really?

cottonseed• 7 years ago

When I hear about rewrites, I always think about this comment:

https://news.ycombinator.com/item?id=11554288

itronitron• 7 years ago

So true, the opportunity to leave things better than as you found them is one of the primary rewards in fixing bugs in existing/old/legacy systems.

nemothekid• 7 years ago

A pattern I've started seeing with most "bad" rewrites is that "bad" rewrites are probably usually motivated by burnout than actual architectural problems. People get tired of working on the code, and they would rather be working on something else hence "let's rewrite".

When you hear about "good" rewrites it's almost never framed in a way that is "the old code is bad and complicated." It's usually driven by an architectural need - for example it started in Python, but now we have scaled up and in order to avoid $1MM/month a rewrite in Go/Rust/C would be beneficial.

Or, in the case of changing marketing needs, it can be "we now better understand our customers and a rewrite can help us provide a more stable interface and help us iterate faster". However in this case a full 100% rewrite is never the case and it's usually done piecemeal or by the new system simply proxying requests to the old system.

When the excuse is simply the "code is a mess", then not only are you admitting that the original team failed to design a maintainable system, you are also trusting them to not make the same mistake again. And if their whole motivation is "the code is a mess", they will probably fuck up the redesign, as messy code is a symptom of bad design but not the root cause. If the root cause is burnout, and that isn't addressed, the rewrite may be just as bad.

rightbyte• 7 years ago

Iterational design is a thing.

If the same team that did the crappy code gets a chance to learn from the mistakes and rewrite it part by part now when they have the whole picture, it's easier and the end result might be mantainable and nice.

marcinzm• 7 years ago

I've heard that the worst code is the 2nd version. You try to fix everything from the 1st version (real or perceived) and over correct as a result.

legulere• 7 years ago

Also architectural needs sometimes are not really needs. E.g. you have a program for dos and think that you need a complete rewrite for windows. Then someone heroically ports over the dos program to windows, because the programs are not compatible. You end up with two programs to maintain.

skybrian• 7 years ago

Also consider how much time you'd lose by not forking a well-designed, open source codebase.

apo• 7 years ago

Consider refactoring before taking a step to code rewriting:

Refactoring is only an option for anything but a trivial code base if a suite of automated tests have already been written.

The article makes no mention of tests. I suspect the reason is that there were none. Refactoring would most likely have been impossible until an automated test suite had been written.

Many developers are confused about the purpose of an automated test suite. It's not to make your designs better. It's to allow you to refactor the spaghetti that will inevitably arise.

Ma8ee• 7 years ago

That is just not true. There’s always the option of refactoring, make the necessary changes and manually test the system. Even though I’m a big fan of automated testing (not only unit tests), most codebases don’t have complete test coverage, but we still refactor and change the systems.

hpolatyuruk2• 7 years ago

The article is not about refactoring. It's about rewriting the software. So that's why I didn't mention about refactoring in details. Automated testing is part of the refactoring. You will change a small amount of code then run tests to see it works as expected. This is how refactoring works. No need to mention abou that.

kstenerud2• 7 years ago

I've been involved in 3 separate rewrites in different industries, at companies of different sizes, over the course of 20+ years.

Not one of them went well.

I've also been involved in maintaining legacy code during that time.

The code sucked, and changes were slow, but it went a LOT better than the rewrites did. We actually shipped things.

yibg• 7 years ago

The problem is it’s not linear. You can ship until you can’t (or can but very very slowly). It’s taking all of the hit now or a little at a time until things completely grind to a haunt.

I’ve been involved in a few rewrites as well. Some went well and we had a noticeable productivity gain. Some went badly and we were stuck for a long time fixing bugs and corner cases we had already addressed in the legacy stack. It really depends on the situation.

gchamonlive2• 7 years ago

what motivated those rewrites?

kstenerud2• 7 years ago

One was pure vanity. We replaced a perfectly functional codebase in order to support a bunch of new functionality more easily. In the end, what we actually did implement could have easily been done on the old architecture.

Another one was a 15 year old codebase. We were also switching to Java. The process had taken 3 years by the time I arrived, and it was eventually canceled. They'd brought in a bunch of consultants (I was one of them), and everyone had their own idea and their own architecture for their parts. The pieces didn't play well together.

The 3rd one was just your standard rewrite of a spaghetti codebase. I still think it was the right choice to rewrite, but it took too long and the project was canceled.

bborud3• 7 years ago

I have several experiences with both bad and good rewrites.

A common pattern among the bad rewrites is that they were too late. Rewrites should be early in a project. Expect to do perhaps 2-3 rewrites at the beginning.

Another pattern is that the team was too large. To many chefs and we ended up with an incoherent mess.

The good rewrites I’ve done were gradual. We agreed on the target state of the code and on the intermediate stages. Perhaps you could call these “big, slow, refactoring”, but they were in fact rewrites.

A couple of rewrites were so successful they changed the rules of the game entirely.

People who say rewrites are either all good or all bad are wrong every time.

goto114• 7 years ago

I don't think anybody is opposed to rewriting modules when necessary. The discussion is around "ground up"-rewriting where the whole code base is discarded and starting again from scratch, like how Mozilla did.

You can perform a long series of refactoring so you in the end have rewritten everything. The important difference is if you in all the intermediate stages have a working program with all tests passing.

curtis• 7 years ago

As a general rule you shouldn't rewrite everything from scratch. This does not mean that you shouldn't, on occasion, rewrite some parts of your system. Rewrites are expensive. On the other hand some code you think is bad really is bad, and it will be a net win to rewrite it.

To make a call like this, you really do need a pretty good understanding of your codebase, your business, your testing exposure, the capabilities of your team, and probably some other stuff as well. In reality, your understanding is never going to be as good as you really need it. This doesn't mean you should never rewrite any portion of your codebase, it just means that you really need to reduce the scope a lot to reduce the risk to a manageable amount.

giancarlostoro• 7 years ago

At my last job all I ever did was rewrite software from scratch. Sometimes the technical debt has led you beyond a point of being able to continue using the solution you have. For example Flash (soon to be ditched by mainstream browsers!) and Silverlight (no longer supported on modern browsers), and heck Java on frontend as well. I rewrote an entire Silverlight application to HTML5 and JS for the frontend and the backend with ASP .NET Core.

Then I also rewrote a Java servlet that I never directly touched the codebase for but I was told you needed a special VM just to get it to compile, I rewrote it in Python using CherryPy. Sometimes the technical debt is not worth keeping around. I sure could of modernized the Java but Pythons a lot more approachable to the rest of the team and anybody can understand what is going on at a simple glance. I can run Python on Ubuntu without a hassle since its out of the box (our main OS in that office).

gfodor2• 7 years ago

Turning one piece of software into another is something which offers a spectrum of techniques which range from micro-refactors up to full rewrites. If you find yourself reaching for the full rewrite option, you should realize you have suddenly reached to the very far, far end of the spectrum of options and are basically taking an extreme viewpoint. If you haven't gamed out and really tried to rule out options across the entire spectrum, you've failed -- not just because you've been negligent in terms of planning and trying to make the best choice for your organization/team, but also because if you do end up doing the rewrite, you've made a specific choice that, unlike most others, infuses a large amount of existential risk to the organization and product.

In the case of a full re-write, there really must be no other way to accomplish the goal. How many rewrites happen when there was literally no other way? Engineers seem to have a bottomless well of creativity when it comes to solving problems, but that creativity often evaporates when focused on the problem specifically of "How the hell do we get from A to B without a rewrite?" No pain, no gain.

garganzol• 7 years ago

I'm pretty sure I know the exact product being described. It is Anti-Malware by Malwarebytes.

Being a customer for many years, I saw the downward trend and it corresponds to that described in the article, both chronologically and by meaning.

P.S. I can add up a lot to this story from a customer's point of view. Let me know if you are interested, I will continue in a comment below.

rwallace• 7 years ago

Yes please!

novok2• 7 years ago

I've been through successful rewrites. I think rewrites make sense when your product has scaled very fast with 50x more engineers or 50x more demands on it, so the context in what was made doesn't make sense anymore. You need a faster language, a stricter architecture and code discipline to deal with +100 engineers working on one product vs the previous 3, etc.

Rewrites in that case is more like your old wooden 4 story office is too small, so now you need to build a skyscraper out of steel and concrete. While the steel concrete structure is being built, you still keep an maintain your old office since it's running your business, and you have the budget to simultaneously maintain both. Steel structures require different building techniques than your wooden one.

That kind of hyper scaling is really rare outside of SF venture capital companies, so in most cases, you shouldn't do rewrites.

beachy• 7 years ago

It's also possible that things that grow 50x tend to be newer, meaning less gradual accretion of cruft in the code. Being newer also means more remaining institutional knowledge in the organization about the technology.

For 50x growth, the new thing we are thinking of rewriting probably also has some simple core purpose (like some kind of social network), making it easier to grok the existing code and indeed to rewrite it.

By contrast a decrepit old payroll system will not have the growth dynamic, and will not be at all simple to rewrite - the code will be packed with small fixes and strange little features that are actually critically important, and that it would be disastrous to omit in a rewrite.

These tie in with your last point.

jcoffland3• 7 years ago

1. Are you ready to throw away all that knowledge?

This is a massive fallacy. Rewriting code does not have to throw away knowledge. The old code is a specification for the new. You just have to read it.

I've been involved in several successful rewrites and I always use the old code as a precise description of how the software is supposed to work. You have to take the time to understand it. And if you don't understand the old code, you really shouldn't be rewriting it.

AnimalMuppet3• 7 years ago

The code as "precise" description: The problem is that you can't tell if that weird "if" is correctly handling a corner case, or incorrectly handling a corner case, or is now a "can't happen" left behind after something else changed. You can't tell which from the code.

That means that tribal knowledge is crucial for a successful rewrite. And that means that you need to do the rewrite while you still have that knowledge available.

jcoffland3• 7 years ago

I don't agree. The code may not do precisely what was intended but it is deterministic. If you understand the language, you can understand the code.

switchbak• 7 years ago

This is a good reason to have decent, intent revealing tests. Even if the code descends into a ball of mud, you still have the tests characterizing what the function of the system ought to be, allowing a rewrite with more than just blind faith and seat of your pants optimism.

Then again, code bases with suites of extensive tests probably tend to not be the real worst cases. But if they are, at least you have the tools to do lots of iterative/evolutionary refactoring. It's amazing the radical transformations that crappy code can go through when you gain confidence from good tests.

Having done huge multi-year refactoring efforts in both scenarios, I know which one I'd prefer.

scottrogowski• 7 years ago

I've been a developer for almost 10 years and during that time, have been involved in small rewrites and major rewrites. None of them have gone well and there wasn't a single one that we didn't regret starting on. What you think is a mess of legacy code is often less messy than you think and a lot of the odd decisions were often made for very good reasons. When you rewrite a codebase, you often tend to over-engineer it for the vague goals of being "scalable" and "clean" but, while clean on the surface, it often comes out far more complicated and with the a similar number of bugs. Another way to look at it is you're taking what was previously an agile approach of iterative development and replacing it with a waterfall approach which sees a grand end goal (parity with legacy code). Software is complicated and there's no silver bullet. Adopting React over jQuery will not reduce complexity as much as you think. Adopting microservices over a monolith will not reduce complexity as much as you think. The only thing that will ever reduce complexity is careful iterative development of good abstractions which are inserted into the codebase one-at-a-time.

jandrewrogers• 7 years ago

Rewrites, as opposed to refactors, are warranted when they address fundamental architectural limitations that hinder product success. There is a notable exception for cases where the software architecture effectively is the product and customers rely on observable behaviors good or bad e.g. RDBMS. Poor design is not the only source of architectural problems and it is not always avoidable; maintaining unused flexibility and optionality in architectures is often extremely expensive.

In my experience, few companies rewrite systems with legitimate architecture issues. It is vastly cheaper in the short-term to redefine the business in terms of those limitations, which is what most actually do in these cases. The risk to this strategy, which I've seen manifest many times, is that a competitor without these limitations can change the expectations of the market and thereby render your product obsolete in surprisingly short order.

Rewrites are always extremely expensive, along many dimensions, but they are also sometimes unavoidable and can lead to much greater product success than without. It is much more complex than the mere state of your code base. The calculus for whether or not it is worth it doesn't lend itself to simple analysis.

bborud3• 7 years ago

1) Rewriting code is rarely, as the phrase suggests, about doing all the work over. Usually it is about restructuring the architecture and then adapting the old code gradually.

2) Throwing away code is not the same as throwing away knowledge. Provided that you didn’t lose the entire team, the knowledge is still there. The code is just the corpse of knowledge and it is worth just a fraction of what the team that wrote the code is.

A rewrite is a good opportunity to revisit and review that knowledge.

3) With experience you learn to balance two forces: perfection and pragmatism. You should strive for perfection in everything you do, but you have to demand of yourself that you can release in a reasonable amount of time.

4) Every good software project I’ve been on has viewed the code as something temporary. This forces you to think about how you will replace every bit of the code as you write it. From this, workable architecture emerges unless you succumb to perfectionism.

From your description your CTO’s instincts were right, but he might be missing the backbone to put you in your place.

Your team needs a more experienced lead developer to help you guys mature as developers. And you need to pay less attention to articles and blogs, be less worried about what others think and ready to learn from someone more experienced. That’s going to be hard since we programmers tend to have huge egos. Even when we’re inexperienced.

revskill• 7 years ago

My favorite strategy for code reuse is, make API configurable. That means instead of coding, you configure the system as much as possible.

Only the core , which is the implementation details could be changed without affect the rest of system.

One benefit/downside is that, whenever you change the core, or everything else works at the same time, or all thing breaks.

anaphor• 7 years ago

Am I the only one who will do an initial "proof of concept" implementation of something that I intend to rewrite in another language? E.g. I will write the "first draft" in Python or Racket because I know the language won't get in my way, and then if it needs to be redone in a lower level language or something that I'm less familiar with, then I have a reference implementation to compare it with.

The same concept can even apply without switching languages if you just want a "naive" version that you know is correct.

tolien• 7 years ago

Consider refactoring before taking a step to code rewriting

There's got to be a Ship of Theseus argument here, where over the course of time you refactor the whole thing until none of the original code remains. I guess the difference between that and a rewrite is that after each refactor you should still be able to ship it.

fatbird• 7 years ago

I was the main programmer on a web application with 250k lines of code; it handled a single very complex product selection/configuration process. The time came to expand it to handle multiple products in the same family of products.

We ended up, over six months, doing a complete rewrite that, from the outside, looked like refactoring (meaning, from the user's/product owner's perspective, the old stuff continued to work as we wrote the new stuff beside it, switching over as we went).

We launched on time and on budget. My last week was a death march but that was it. There was now only 50k LOC. Where the previous version was tightly coupled to the product specification, we now had an engine that could perform the same selection/configuration process for any product that could be articulated in the new DSL we'd developed; as a bonus, the specifications were standalone modules so we could use them in completely separate web apps.

The whole time we were aware that we were breaking the 'never rewrite' rules spelled out by Spolsky and others, but we were successful, the codebase took a huge leap forward both in speed of development and maintainability, and a subsequent project to add another set of products took half the time that was estimated.

I don't know what we did right, exactly, except perhaps that our rewrite was never about burnout or annoyance at the existing codebase; our decisions along the way were always justified by what we had to deliver and a reasonable feeling of an experienced dev that refactoring the old code would take more effort than starting over.

I'm not sure I buy the articles rejection of the rewrite they did, because he frames it entirely in terms of delay in updating and adding features, without seeming to consider that this was just legitimate technical debt that they had to pay off. The old code got them into a position where they could rewrite it. How many startups fail because their attempts to get it right the first time prevent them from ever launching and getting the customers that will pay to fix it later?

beagle3• 7 years ago

Most rewrites are unjustified, either because the old code is good enough, or because the guys in charge of the rewrite cannot deliver better (whether or not the old code is good enough).

But some are.

Arthur Whitney creates each version of the K language from scratch (he is now at version 7). They are not backward compatible, but every single one so far has been better on the metrics that he cares about (succinctness, speed, and revenue).

Everyone says that the Mozilla->Firefox rewrite was unjustified, but ... the Mozilla codebase looked like a dead end to too many people working on it (from first and second hand accounts I heard), it's not clear that it was viable to continue with the old codebase -- and, even if it was, it is unclear that Mozilla would have been better off doing that: Microsoft was the unstoppable juggernaut playing dirty. IE won not based on technical merit (although it actually WAS the better one come version 4) - but arguably because it was integrated into the OS. In fact, it is quite likely that we would not have firefox today at all if they did not start the rewrite back then.

C# is essentially a rewrite of Java; done for legal reasons rather than technical reasons, but IMO the result is a much better technical solution than trying to retrofit and comply with the (legal) legacy of Java.

But ... yes, most rewrites are unjustified and often end in failure.

kujaomega• 7 years ago

I recommend reading about Domain Driven Design. For example reading the book: Implementing Domain-Driven Design by Vaughn Vernon

Lessons learned from rewriting code

Related Stories