Every app has a scary basement—dark old mysterious code vital to its operation

http://miksovsky.blogs.com/flowstate/2010/09/every-app-has-a-scary-basement.html

by Terretta • 16 years ago

74 23 16 years ago

miksovsky.blogs.com

hysterix• 16 years ago

"When I was first exposed to this reality, some friends on the nearby Microsoft Word team shared stories about their app's own scary basement: a routine called FormatLine. Given a point in a document and a column width, FormatLine would lay out the next line of text at that point. As I heard it, this routine had evolved into a handful of functions that were each thousands of lines long. Developers assigned to descend into the depths of FormatLine were treated with the same respect and concern as spelunkers attempting to reach trapped miners."

This paragraph is hilarious and horrifying all rolled into one. That is not my idea of a fun project.

city41• 16 years ago

I had a friend who was on the Excel team, he told me similar horror stories about some of the very old code that is still in Excel to this day and about as unmaintainable as can be.

Replaid• 16 years ago

I think this puts into words the potential benefit I see in approaches like Google's GWT, which uses Java's ability to describe strict separations to separate "hairy cross-browser engineering fiddliness" from "view intent".

I did say "potential". :-) And the same lesson definitely applies beyond the need to separate UI concerns, but that issue seems to get the tangliest the fastest.

garyrichardson• 16 years ago

Yeah, I think we all design systems that try to prevent shooting ourselves in the foot, but the need for performance usually breaks this. From the article:

To get the most performance out of the PCs of the day, I believe he had the register itself more or less directly read and write transaction data from disk.

He's implying that the normal view/business logic separation wasn't fast enough.

jng• 16 years ago

This is so true...

ekidd• 16 years ago

You _can_ turn this kind of scary "basement" code into something that's maintainable, but it's often a multiyear slog. I've salvaged C++ horrors that were virtually all basement code, intricate and fragile and dating back to Win16 code written in 1993.

Here's a recipe that may help you get started on your adventure:

1. Work incrementally. Never try to fix the whole program at once. Overhauling the whole program inevitably introduces lots of disruption for users and delays the addition of new features. This will not be appreciated.

2. Make sure you have tests. Lots and lots of tests. Any time you touch a module, write some tests for that module.

3. Build better module boundaries. This can be a mechanical process. For example, to separate disk I/O code out of a checkout register, declare a RegisterIO class. Every time the GUI code touches the disk, add a new method to the RegisterIO class and move the code there. You'll wind up with a _really_ ugly RegisterIO class containing a lot of ad hoc IO functions. But the RegisterIO class will improve with time.

4. Reimplement small modules one at a time. You may want to run the old and new versions of a module in parallel for a while (I once had a program with two different text rendering subsystems), or run the same unit tests against both the old the new module.

Eight years of work later, you'll have a reasonably shiny new program with some odd architectural decisions and a really ancient logging subsystem.:-) But at least you will have been shipping new versions and earning revenue every step of the way.

silentbicycle2• 16 years ago

Michael Feathers's _Working Effectively with Legacy Code_ is entirely about dealing with this sort of code, especially on carefully retrofitting tests so you can safely do deeper restructuring.

mminolt• 16 years ago

Everything is fixable. It's just a question of time, process, testing, re-testing, etc. Is it always worth it? No.

stan_rogers• 16 years ago

There was a great "scary basement" story posted here about three years ago that seems to have gone virtually unnoticed:

http://news.ycombinator.com/item?id=85587

If the name seems familiar, well, it's either because you bleed yellow or you've heard of CouchDB.

sliverstorm• 16 years ago

Things like this make it seem worthwhile to create a code translator. In the same way the compiler converts the code from horrific basement language to computer language, it would convert the code from basement language to a properly structured maintainable form. Not just re-formatting, but perhaps changing dated methodologies etc.

Kind of like perltidy, but much more aggressive, pruning and re-arranging and re-writing bits as needed.

On the one hand, in theory a computer could never do it as well as a human, and we should learn to code properly. On the other hand, it's a very common problem, and computers are supposed to be time-saving devices above anything else.

DougWebb• 16 years ago

Perltidy just handles formatting. Scary basement code is often poorly formatted and reformatting is a necessary first step in fixing it, but it's a trivial first step. It doesn't address the hard part.

Perlcritic is closer to what you're thinking of; it examines the structure of the code and points out design and implementation problems. It can't fix them for you though, and it only handles common designs that it knows about. It can't judge the quality of a design that it's not already programmed to recognize. Again, a useful tool for fixing all of the common problems, but it can't address the overall design.

The refactoring tools in IDEs like Visual Studio are useful too, but they don't understand your code in a way that would help redesign it. They just automate much of the process of extracting code from one method to another method/class. Useful, but not redesigning your code for you.

Ultimately, it takes an intelligent software designer to do this kind of work. Maybe artificial intelligence will handle it some day, but that's not coming for a while.

tomjen3• 16 years ago

Writing such code would, unfortunately, be non-trivial (in the geek meaning) at best. Most likely undecidable.

enneff2• 16 years ago

"To get the most performance out of the PCs of the day, I believe he had the register itself more or less directly read and write transaction data from disk."

I don't buy this excuse. It just sounds like bad code to me.

jerf3• 16 years ago

Times were different back then. The article says the original control dates from 1990, and wikipedia confirms that Windows 3.0 came out in 1990 so it is plausible. The 486 had just been released in the previous year, so it was the New Hotness and not very many people had it, so you'd be looking at an install base of mostly 80386s with a substantial contingent of 80286s that your program still needed to run on with acceptable performance. Wikipedia says for the 80286 you're looking at a top-end of 12MHz, with 6 and 8 MHz in the field. Since this is a Windows 3.0 program, you had a minimum of 640KB on the system (though of course you can't use it all).

So this is a program that has to run with reasonable performance on a 12MHz 80286 with 2MB of RAM. Are you sure you wouldn't consider having your UI read and write the disk directly? Separation of concerns is a modern luxury. And remember, you're using compilers from 1990, too. Don't expect great inlining. (Besides, only profligate memory spendthrifts would inline a function.)

(Edit, later followup: In fact, "separation of concerns" is one of the major sources of the "bloat" people complain about when they examine modern OSes. Much of the "non-bloated-ness" of earlier generations of OS doesn't come from programmers caring more about their craft, it comes from programmers having to stuff programs into things that can barely hold them and building "clever things" that we would today consider monstrosities like UI widgets that read and write directly to the disk. I am unsympathetic to accusations that, say, a modern KDE desktop is "bloated".)

caf• 16 years ago

It was a rare 286 indeed that came with 2MB of RAM!

prodigal_erik• 16 years ago

To be fair, memory bandwidth was relatively better and pipelines were nonexistent, so there was little reason to inline a function unless the call setup was literally longer than the body.

My favorite monstrosity was the http://en.wikipedia.org/wiki/Overlay_(programming). When your OS couldn't page functions from disk for you, you damn well did it yourself. They even started building it into linkers....

demallien• 16 years ago

In fact, "separation of concerns" is one of the major sources of the "bloat" people complain about when they examine modern OSes.

Um, what is your source for that assertion? From where I'm standing, separation of concerns is a technique which favorises the re-use of code, and hence the reduction of bloat.

enneff2• 16 years ago

"So this is a program that has to run with reasonable performance on a 12MHz 80286 with 2MB of RAM. Are you sure you wouldn't consider having your UI read and write the disk directly?"

I wrote code for those machines, and no, I wouldn't write my UI code to read and write the disk directly.

"Separation of concerns is a modern luxury."

This is a myth. Merely an excuse for poor API design. Proper abstraction doesn't mean inefficiency.

Your last paragraph (the "Edit") is almost offensive. The cause of bloat is not separation of concerns. It's that people just don't care about writing efficient code (or they just don't know how). Did you ever write code in the era you speak of? Because your statement comes across as ill-informed conjecture.

danbmil99• 16 years ago

We used to call this "Hemoglobin". It's something scary complicated, created by elves (as far as you're concerned); no one knows how it works, but it does (barely), and only if occasional offerings of live virgins are made at its altar.

Another appropriate term is "Happy Fun Ball" (google it)

Every app has a scary basement—dark old mysterious code vital to its operation

Related Stories