Technical Debt Doesn't Disappear

http://caseysoftware.com/blog/technical-debt-doesn039t-disappear

by tswicegood • 15 years ago

18 19 15 years ago

sp_2• 15 years ago

I used to work for a startup that rewrote its products all the time. It was a huge mess with all the consequences outlines in the article.

The problem was not the rewriting itself but the lack of understanding how things went wrong the first time. From my experience it is absolutely mandatory to have many deeply introspective and self-critical thought sessions to figure out how you got into the mess. Only if you have clearly identified what went wrong in the past, you will be able to successfully rewrite a project and avoid the mistakes.

For some of the products we did this and rewriting was a huge success. For some others we did not and the rewrite turned out to be just as terrible as the original version. Then we rewrote it again and it was still a mess.

caseysoftware2• 15 years ago

Either way... the original problems don't just disappear. They're with you all along until you completely replace the product.

And even then, they don't disappear. Windows 98 is 12+ years old and it's still lurking around out there. Vista was only the main version of Windows for 2 years and I bet it will haunt our support queues for another 5 years at least..

stcredzero4• 15 years ago

Assume you're going to get it wrong, and leave yourself an out.

Often the cost of refactoring is much less than the cost of a rewrite. If this isn't the case, then I suggest you look at your language/toolset/coding standards.

Is it possible to get yourself to the point where you no longer have to pay the cost of a complete rewrite? It might even be possible and worth doing a rewrite for the purpose of avoiding future rewrites. (And if you say this is like "a war to end wars," I'll just say that's cute, but wrong.)

Was there ever a "Five Whys" asked about the rewrite?

sp_2• 15 years ago

The one product I was mostly involved in needed to be rewritten because it was the result of a quick proof of concept hack that kept growing because people actually bought it. There was literally no separation between GUI, model, or DB backend. Adding to the code was like swimming through molasses.

I took the product over as lead developer and saw no way to save it. It helped that only maybe about 10% of the features we wanted to have were implemented, so two of us managed to do the rewrite in less than six months.

The main focus of this rewrite was indeed to make sure that we would never have to rewrite the product ever again. We kept a very close eye on identifying what went wrong, what design principles can help us to avoid doing things wrong in the future, and how to design for extensibility considering that 90% of the features still have to be added.

In the end the rewrite turned out to be very successful. Until the day I left the company (three years after) no rewrite was ever necessary again. Rather, we managed to keep up the good principles laid down during the rewrite so we could just work on the code incrementally. The product was also the commercially most successful piece of software the startup sold.

zdw• 15 years ago

Anyone know of a link to the mentioned talk? The one given ( http://joind.in/talk/view/2418 ) is just the conference description.

romaniv• 15 years ago

Do we really need all these articles that urge people to never, ever rewrite anything? (Classic example being Joel's article, which also uses Netscape as an example: http://www.joelonsoftware.com/articles/fog0000000069.html .)

Managers and business users hate rewerites as it is. That's part of the reason there is so much horrendous legacy code out there. Often, people who have the authority to make decisions about rewriting something don't interaface with the code, don't understand undetlying issues, don't see what was changed in technological context since the code was written.

In my own experience, most of the rewrites were totally worth it. It wasn't just because the code became better, it was because the understanding of the problem has matured. People were able to understand the problem better, so were were able to write better code to solve it.

j_baker• 15 years ago

I read it as saying "Spend a little extra time doing things the right way, and maybe you won't have to rewrite your code from scratch."

Dylanlacey• 15 years ago

I've found the opposite to be true: Managers and Business Users LOVE the re-write, because it's easy to tell themselves the calming little lies.

Lies like "We already know what it should do!" and "We don't need to maintain the legacy, your issue will be fixed in the New System" and "There won't need to be any changes to business processes!"

I have never seen a re-write done with a greater understanding of the problem. In every case I've observed, the non-developer stakeholders start insisting on features working exactly as before OR start adding new, wonderful features from the get go. It results in a mess that solves NOTHING.

The debt metaphor is very apt. You can declare bankruptcy on your technical debt and walk away... But your tech credit history (IE your business systems) are a mess, your creditors (users) start to hate and mistrust you, and starting anything fresh becomes complicated beyond belief.

Re-writes suck. They SUUUUUCK. And people persist in doing them. That's why we keep needing these articles.

agentultra• 15 years ago

Legacy systems are hard. Being able to work effectively with them is something they don't really teach you in school. Probably because it's entirely frustrating, complicated, and a big turn off even for seasoned programmers. It's a bit of a black art and there's little joy or glory in being good at it. Except perhaps that the job market for your specialty is pretty small and will always be in demand.

For a good read on the subject, try "Working Effectively with Legacy Code" by Micheal C. Feathers.

I prefer the term "technical baggage," over "technical debt." This is mainly because debt refers to an economic means of borrowing a sum of future purchasing power. This breaks down when referring to the cost of engineering decisions. The cost of maintaining legacy systems tends to increase with time and you cannot calculate any sum to borrow based on assets or secured revenue. The decisions you make now will haunt your business for a long time to come. Poor technical decisions are simply something you will have to learn to live with.

When dealing with legacy systems from a business perspective, the question one has to ask is, "how valuable is it to maintain and enhance the existing system going forward?" In other words, when will the costs of continuing development of your legacy system outweigh the revenue it generates? For large systems found in banks and insurance companies it can be a very long time. For small businesses it is much shorter than you think.

FWIW, it seemed to me that Netscape simply made the decision to start from scratch too late. The revenue they were generating from their old code base had trickled off too long and dipped below their operating costs before the warning signs went up. I don't think it makes a compelling argument against re-writing.

As someone supporting a legacy system, it sometimes makes sense to re-write the system. Treat the existing system as a prototype and derive the minimum viable product from it to start with. Spin it off as a new product, set an shut-off date for the old software, and convert your customers. Obviously this doesn't work for every scenario; so YMMV. Just be smart about it and use evidence to back up your assumptions. Sometimes simply writing a legacy compatibility layer and porting the most painful bits of code slowly is the key. Re-writing certainly isn't for everyone.

stcredzero4• 15 years ago

I prefer the term "technical baggage," over "technical debt."

Yes, but sometimes, this stuff actually compounds!

sjmulder• 15 years ago

This hits home. I was brought into a company to work on a product together with an other new hire, and the company decided that we would go and do a rewrite – for this decision I was partially to blame. Six months later we were stuck with a failing ‘classic’ site that had barely been maintained, angry customers and nothing to show for it. Lessons learnt.

stcredzero4• 15 years ago

It's really about sustainability. Technical debt isn't sustainable. Getting into a situation where you're afraid to fix the old thing because of opportunity costs isn't sustainable. Most often, this is because the cost of testing is too high and/or the cost of the bugs that would probably happen/escape testing would be too high. Quick fixes that get you through the next month at the cost of more technical debt going forward aren't sustainable.

Sustainable practices are smart practices.

laf2019• 15 years ago

At my current job I was tasked with making a new framework and transitioning our code over to it. The reason that we decided to actually work on technical debt is because we are growing very rapidly and the old setup is too slow so the customers are actually affected. It was extremely useful to have every technical stakeholder in the design discussion to figure out what features we really need, where we went wrong with the old design, and how to transition properly without spending 10 months. So far so good, we are transitioning a bunch of our code over to the framework and it seems to be working well. Im sure mistakes will be made along the way, but planning with people who made the original mistakes (and subsequently have manned up to them) should help prevent remaking original mistakes.

tomjen3• 15 years ago

> The scariest problem is our project_list_data function with Cyclomatic Complexity of 145 million.. which is down from 195 million. It "passes" data in via a series of globals and "returns" data the same way. Don't worry, we're working on it.

Is this meant to be sarcasm? Because if not, thats an insanely huge function. I imagine most compilers would choke on it...

caseysoftware2• 15 years ago

No... it's not sarcasm at all. It's a ridiculously complex function with nested-nested-nested conditionals in nested-nested loops.

It makes puppies cry.

kirbman89• 15 years ago

A book that I would recommend for refactoring object oriented code (Java in particular) is Refactoring-Improve the Design of Existing Code by Martin Fowler (a friend to the Go4). Written well and in a simliar format at Design Patterns-Elements of Reusable Object-Oriented Software by Go4 (Gang of Four).

jtbigwoo• 15 years ago

I think that architects and developers often forget the basics software development when planning rewrites. Because they've been working on the product for so long, they think the normal rules of working iteratively and getting feedback don't apply. They try to replicate the entire existing system at a point in time, turning the rewrite into a waterfall project. Then they're surprised when things they fall behind and can't adjust. You're better off in almost all cases to break it up and approach the problem as refactoring instead.

When I was an "e-business" consultant, we had a saying: "Six month projects take nine months, twelve month projects take eighteen months, and two year projects get canceled." If your rewrite is projected to take over a year, you're in huge trouble.

bod• 15 years ago

This is rather concerning the presentation of the text: Did anyone else find the black font of this size on green background disturbing for the eyes? Selecting the text helps, though...

Technical Debt Doesn't Disappear

Related Stories