Embrace technical debt

http://startuplessonslearned.blogspot.com/2009/07/embrace-technical-debt.html

by nreece • 17 years ago

68 27 17 years ago

startuplessonslearned.blogspot.com

wynand• 17 years ago

This post makes a good companion to a post of a couple of days ago: "Why Do We Keep Building Tightly Coupled Software" (http://news.ycombinator.com/item?id=705863).

A common theme in the comments to that story was that there are costs associated with loose coupling. Not making loose coupling a first priority means that one will incur some technical debt; but given that one gets something that will work sooner, this is often a good call.

Does anyone know of any software teams that explicitly talk about their technical debts and that have plans for "downpayments"?

sokoloff2• 17 years ago

We do. Our [internal] wiki article on technical debt explicitly covers examples, guidance on when to intentionally take on debt, how to document and measure it, different "grades" of it, and how to plan to retire it.

There are then per-application technical debt pages for specific details on our "portfolio" of debt. (We're also a fair bit larger than most startup dev teams, so having this level of formalism is more appropriate in our case than say just a whiteboard somewhere...)

darkxanthos• 17 years ago

I would love to know your formal approach (or as close to it as possible). I personally see technical debt as debt to be leverage but I lack the means to track it so that we are always aware of how much debt we have.

kreem• 17 years ago

I think this is a good article, but most times companies do not even know they are incurring technical debt, and that is the first step before you can start working with technical debt in an intelligent way.

jasongullickson• 17 years ago

Weird, I was just trying to explain this to a manager yesterday, I need to go look for hidden microphones...

...I've tried in the past to explain this to others using retail "shrink" as an analogy. For those unfamiliar with the term, as I know it, the idea is that you assume a certain percentage of loss (in my experience, as shoplifting in a music store) and by writing that off, it allows you to focus your energies where they have more impact on profit. For example, placing the lowest-margin items in the most vulnerable parts of the store to deter theft of items where you really make money.

I'm glad someone took the time to write up a better explanation that I can now refer to.

lallysingh• 17 years ago

LiveScribe pens.

jon_dahl2• 17 years ago

I tell my clients that cutting corners is "an hour of savings today, 5 hours of cost tomorrow". I've worked on too many projects that raced to cram in features by a deadline, and ended up with a sloppy app that took forever to maintain and made developers miserable. Far better to take a measured pace and ensure that rapid development can happen in months 12-24, not just months 1-3. Especially in startups - startups are a marathon, not a sprint.

Of course, cutting corners isn't the only kind of code debt, and the author makes some good points. If you're spiking something or prototyping it, and intend to rewrite or throw away, technical debt might be a good tradeoff.

I also take issue with this:

Early on at IMVU, we incorporated in tons of open source projects. This was a huge win... The downside was that we had to combine dozens of projects whose internal architectures, coding styles, and general quality varied widely. It took us a long time to pay off all the debt that incurred – but it was worth it.

The more you can offload to open source libraries or third party APIs, the less your application has to do, and the less you have to maintain. If the library or API sucks, then sure - you're incurring technical debt. But if the library/API is good, or you can contribute to make it good, then it's a win all around.

swombat4• 17 years ago

I also take issue with this:

I think that's more or less exactly the point he makes. Some external dependencies are good, some are dreadful. At some point, you need to address the bad ones.

I tell my clients that cutting corners is "an hour of savings today, 5 hours of cost tomorrow".

Re: cutting corners, it really depends what you mean by "cutting corners". There's a whole continuum of code quality from "omg this sucks I want to kill the person who wrote it" to "this is the most beautiful piece of code every designed by man. Anything short of the latter is effectively incurring debt. I don't think Eric Ries is advocating dirty, nasty coding. Instead, I'd say he's suggesting that you should be comfortable with not writing the best possible code, rather than always feel like you're not doing well enough.

Tamerlin• 17 years ago

I don't entirely agree with the last paragraph. IMO you should ALWAYS write the best possible code. Any assertion otherwise is equivalent to saying you should be willing to put bad code into production.

The trick isn't in reducing code quality to save time. The trick is simplicity. Too often developers equate quality and complexity; the two are orthogonal.

A simple architecture with high-quality code will take far less effort to maintain than a complex architecture with high-quality code OR a simple architecture with low-quality code.

In my experience, the vast majority of the technical debt I've seen has resulted from building complex solutions to simple problems.

Keeping the solutions as simple as possible to fit the needs of the project is a very good way to keep down technical debt AND facilitate rapid development.

nostrademons3• 17 years ago

Sometimes it makes sense to take that hour of savings today at the cost of 5 hours later, though. I've been on projects where we knew a competitor was coming to market soon, and sacrificed a bit on code quality to beat them. We ended up launching 2 weeks before they did, which meant that when they finally did launch, the press was all like "But can they compete with Company X, who launched a similar feature two weeks ago?" Doing things the right way would've added about another month and a half - if we'd paid that cost up-front, the press reaction may've been "But their relevance may be fading, because Company Y launched similar features a month ago."

jon_dahl2• 17 years ago

I agree. You really have to be disciplined to do this, though, because there is always something in a startup that needs to be done right now. Rushing for a month and then taking two weeks to refactor/improve code is fine. But doing the same for 12 months is a big gamble.

apinstein5• 17 years ago

Technical Debt is a broken metaphor. It's almost right, which is why it resonates, but it's a bad mental model for the problem it tries to describe.

There are real negative consequence of applying a "broken mental model" -- you end up doing the wrong thing when you live by the model!

Why is technical debt a broken concept? Real Debt is incurred proportionally to the asset bought with it. In technical debt the debt is proportional to the trouble it causes, not the value delivered. Also, it implies that the only way to get rid of the debt is to make a big investment. In reality, you can just throw away code and the debt disappears. Not analogous with real debt!

It should've been called technical drag. Properties of drag that are appropriate to the metaphor: - You get value based on the amount of input work, but it places a drag on the overall progress towards the goal (support load, architectural coupling). - You can reduce the drag of an existing component by investing to refine it. - New components typically add more drag than existing ones. - Little bits of drag here and there add up to a big problem - Removing unused functionality reduces associated drag without major investment

Thoughts?

swombat4• 17 years ago

Agree. In fact, I called it "viscosity" in an article a long time ago, and posited that the rate of increase of viscosity depends on languages, practices, etc:

http://inter-sections.net/2007/12/01/comparing-pieces-of-str...

"Technical drag" is a very good term for it, I agree.

apinstein5• 17 years ago

Interesting article. I think using viscosity to compare languages or frameworks is counterproductive, but the general idea is useful when talking about software architecture and development.

For instance, I used to manage some poorly architected Java apps where even trivial changes incurred painful re-architecting because the app was poorly written. And I've managed PHP apps that are extremely well-organized and are very simple to add functionality to.

The other thing I'd mention is I like your idea about hitting the wall where improvement freezes b/c the implementation time is so great. I think it's a good fit for the technical drag model -- that is the point of terminal velocity!

It's another good-fit for the metaphor; eventually you can only drive forward a product by making the team bigger because your project hits its terminal velocity (or top speed) for the given "engine" size (the team).

tome• 17 years ago

In reality, you can just throw away code and the debt disappears.

Technical debt is closer to this than you might think. This would be "going bankrupt" on that code. It's like having a loan secured on the value of some object. If you want to default on the loan, you lose the object, nothing more. (Although you don't lose your good credit rating as you would with real debt!)

apinstein5• 17 years ago

Um, what? No! This is exactly the type of logic-wrangling that happens with bad mental models.

It is not going bankrupt on that code, When you go bankrupt, there is a huge penalty to your credit! If you stop maintaining code that no one uses, the drag goes away and there is no downside!

Deleting code is fun. Going bankrupt, not so much.

dusklight• 17 years ago

I prefer the debt metaphor versus drag for two major reasons. First when you talk about real physical drag, the amount of drag per object is not a value assigned by a human person. You don't choose how much drag an object has. You can change the shape of the object to modify the drag coefficient but you cannot modify it independently. Whereas with debt, it is understood that this value is something that is your choice.

Secondly, with debt comes the concept of interest, which is the most salient point. The big problem with technical debt is that you can have a small amount and there are essentially no downsides, but the longer you avoid paying for it, the harder it is to avoid creating new technical debt when you write new code, the more technical debt you will incur, and the negative effects start to compound.

apinstein5• 17 years ago

You can't independently choose your "debt" level with software! When you design feature X, it costs you interest in terms of support, refactoring, debugging, etc. If you think you can decide exactly how much that's going to cost you a priori then you should go buy some lottery tickets. Drag is a better metaphor in this case because you can have an idea from looking at the design approximately how painful it will be to support it, but you can't be quite sure until you test it in the real world (ie wind tunnel test for example). You can model/simulate it based on expectations of feature popularity etc, but you won't know for sure.

On your second point, I do agree that there is a strong salient aspect of the idea of interest payments on debt. Like I said originally, the metaphor stuck because it resonates. However, the idea of technical drag has this same exact concept; the more junk you bolt on to the car, the more it overwhelms your project, and your initial F1 racer becomes a UPS truck.

One could make a point about technical debt that technical drag doesn't have, and I'll yield it's important, is that it's pretty tough to have so much drag you can't move. Whereas debt service can flat-out sink you.

However I'd counter that if you aren't moving (ie have customers using your software) then technical debt isn't really a problem because you don't have to do much work to service software with no users. Whereas technical debt implies that you have to pay for the feature even if it isn't being used.

wglb• 17 years ago

This is a good article and it resonates with one of pg's essays in which he says that sometimes in building a project you worry about security later. I think it is also related to the early optimization error. You don't know what you are ultimately going to build until you get feedback from customers/users. So don't overengineer/optimize something that you aren't going to use/doesn't need it.

jimbokun• 17 years ago

"As in many scalability decisions, we’d have been much better off investing in agility, so that we could change the architecture in response to actual customer demand, rather than trying to predict the future."

With a poor design, it can be more difficult to make changes to improve scalability. If you do not have abstractions and information hiding that allow you to switch out algorithms, libraries or systems for ones that scale better, you will need to rewrite a lot more code in order to to scale.

"Yet other agile principles suggest the opposite, as in YAGNI and DoTheSimplestThingThatCouldPossiblyWork."

My concern with the premise of this article is that some might use it as a license for sloppy code. YAGNI and DoTheSimplestThingThatCouldPossiblyWork can be accomplished with code that is concise and clearly expresses intent. In fact, you could argue that these principles demand such code. I've seen a lot of code with many lines that accomplish very little, and that kind of "technical debt" never pays off, in my experience.

Embrace technical debt

Related Stories