The Duct Tape Programmer (2009)

https://www.joelonsoftware.com/2009/09/23/the-duct-tape-programmer/

93 50 6 years ago

rfreytag3• 6 years ago

Duck tape programmers also produce code only one or a few persons can patch. The result is technical debt that lead Netscape to do a full rewrite to produce Mozilla/Firefox.

The truth is in the middle with better tooling to make it easier to design good code and get it working faster with fewer bugs even while at the bleeding edge.

amrox• 6 years ago

yeah sounds a lot like the fabled "10x engineer" to me

KaiserPro2• 6 years ago

Not really.

You're assuming that ductape == bad. What your describing is someone writing bad code, not someone gaffertaping a bunch of modules together to make the _more important_ code work. (instead of re-writing a whole bunch of things to make stuff more "idiomatic" )

This is where innovation tokens come in. You have one innovation token per project, you can spend it on the thing you actually need, or some pointless part of the system that you're bored with maintaining.

rfreytag3• 6 years ago

I mostly agree. This is why I come down on the truth being in the middle.

xnx2• 6 years ago

Netscape might be the worst example for your claim. Read why here: https://www.joelonsoftware.com/2000/04/06/things-you-should-...

rfreytag3• 6 years ago

Quoting from that article (which I remember reading when it came out): "It’s important to remember that when you start from scratch there is absolutely no reason to believe that you are going to do a better job than you did the first time."

Mozilla and Firefox are better products. Many other solutions use their code (e.g. TOR).

All bets are off when competing against a monopolist.

Netscape maybe died because the technical debt forced them to bet the company on making a better product...which they did.

SomeCallMeTim• 6 years ago

I think the extremes are straw men. The choice isn't between "duct tape" or "oven-engineering." It's between poor and good design.

A well designed solution doesn't feel like duct tape, nor should it be "too clever" with a crazy multiple-inheritance architecture that no one but the author can understand.

In fact, it sounds like the extremes described in this article are both crap developers, with different flavors of crap on the menu.

I insist on solid engineering. Nothing that feels like duct tape, but nothing that approaches over-engineering either. Easy to understand and easy to extend to the point of "OF COURSE this makes sense!"

It's not so much "in the middle" as "robust design vs one or another flavor of garbage." A strong designer is needed to get to that point, and maybe duct tape is better for shipping product than over-engineering, so if you HAVE to choose, the duct tape end of the spectrum is better. But even better to have a solid software architect who can come up with a good solution that people can use and extend and understand.

Uhhrrr3• 6 years ago

I would say the extremes are between no design and some "perfect" design.

codr7• 6 years ago

So a pragmatic generalist?

Duct Tape sounds more like what most are unfortunately stuck doing: inheriting this, depending on that, and plugging the whole mess into the cloud.

jacobrussell2• 6 years ago

Still going back to Joel on Software for a good read. My favorite technical blog even though Joel has more or less stopped writing except for the occasional update.

logbiscuitswave• 6 years ago

He has so many classic must-read articles that are just as relevant today as when they were written some 10+ years ago. The technologies may have changed but the sentiments sure haven’t.

jacobrussell2• 6 years ago

I remember reading "The Joel Test" my Sophomore year of college. It really influenced the questions I asked when going through so many interviews my senior year. Not sure how I even found his blog in the first place. Whenever I found out he wrote the specifications for Excel my mind was blown. I think Excel is maybe the most influential/important piece of software ever written.

KaiserPro2• 6 years ago

This article and the one on innovation tokens are what I try to live by.

ipnon• 6 years ago

Being in a job like this makes me feel like I'm in a Dilbert cartoon.

cryptomeme• 6 years ago

As cheesy as it might sound, I found inspiration in jwz's attitude way back in 1998-ish. Strong principles of pragmatic design, speaking your mind, and being able to walk when the bullshit gets too deep are still a winning combo.

Not to leave out Joel! He's an excellent curator of the software ecosystem, a kind of Steve Irwin of software.

It turns out that proper attitude, pragmatism, and getting results are the most durable skills in software: the rest is just tools and process.

LeifCarrotson• 6 years ago

xor the “next” and “prev” pointers of their linked list into a single DWORD to save 32 bits

I suppose that it's true that if you were coming from the item pointed to by 'prev' you could xor your previous address with the linked list pointer to get the value of 'next'. Is that from an actual implementation? Was anyone crazy enough to actually do that?

silvestrov• 6 years ago

Yes, the Z88 used this for memory: https://www.cl.cam.ac.uk/~jrh13/devnotes/all.html#sec218

Another crazy thing in Z88 is that the maximum amount you could allocate in one 'malloc' call was 256 bytes!

benibela• 6 years ago

It can be done even better. I read an article about that recently.

Nowadays pointers are 64-bit. So the simple xoring already saves 64 bits.

But, in most cases allocated memory blocks are close together, so that xoring of those 64-bit values gives a 32-bit value. So if you ignore the upper half and store the result in 32-bit, you basically save 96 bits. For the cases where the upper half matters, you can store it in an additional hashmap.

But it goes even better, if you allocate blocks and store indices. Like if you allocate blocks of 512 objects each. Then it is likely that the next/prev index is also in that block. Then you can store those indices in 2*8 bits (or just 8 bits if xored again?). In case it is in another block, you use the hashmap again. Then you save up to 120 bits.

The question remains, how do you find that hashmap? The article had a really clever solution. You allocate the blocks as 4K-aligned pages, e.g. by using mmap rather than malloc. Then you put a pointer to the hashmap at the beginning of the page, and when you have a pointer inside the block, you can find the hashmap by rounding the pointer down to the previous aligned position.

glangdale• 6 years ago

These are neat tricks, but they are now completely different data structures to the original one (and, IMO, maybe not all that great a data structure, in that it seems to be a festival of special cases and clever hacks). Once you're willing to allocate large blocks of objects, all sorts of things are possible, but you have to be ready for radically different performance properties (not always bad, just... different).

The XOR-trick is curious (if weird) because it preserves most* of the properties of the list structure. Traversal is almost identical to that of the original list in machine terms. Maybe an additional cycle on the critical path; probably not the worst thing in pointer chasing code.

* you do need to double the size of your iterator, so if you were needing to stash large numbers of iterators relative to the number of linked list elements, this wouldn't be a particularly nice trick.

makach• 6 years ago

This is a good article, although I personally think that the terminology "duct tape" developer is IMO a little bit dated...

PaulDavisThe1st• 6 years ago

I have issues with the combining of what to me are entirely different categories of things in the sort of "critique" that "duct tape" programming supposedly represents.

Fine, COM and COBRA are baroque and complex and one can make the case that they merely make "simpler" things that can be done in other, more duct-tape-y ways. I'm fine with dismissing this sort of overgrown, overcooked-before-anyone-even-ate-it API/framework in favor of simpler approaches.

But ... just because Netscape didn't need threads in 1995 doesn't mean that nobody needs threads. And by threads, I'm not talking about "concurrency" in the way that is in vogue today in the context of Rust/Go/JS. Sometimes, you do actually need threads, and you just have to grapple with having multiple execution streams running simultaneously, with everything that implies.

That's just picking out the most prominent example from TFA, but there are other floating around too. Failure to properly identify "baroque over-designed nonsense" from "complex, hard to use but necessary stuff" is a common failure, and an important one to try to avoid.

Also, the "next and previous points xor'ed into a DWORD" hack mentioned at the end? This isn't duct tape programming. It's absurdly context-dependent premature optimization, useful only in insanely memory-constrained environments, and not likely useful in almost any of the scenarios that most programmers (especially duct-tapers) find themselves working.

Also, real programmers don't say DWORD.

Stratoscope• 6 years ago

Also, real programmers don't say DWORD.

That's just silly. DWORD is a macro for an unsigned 32-bit integer. It has been around since the first version of Windows.h, along with BYTE and WORD for unsigned 8-bit and 16-bit integers.

Any old-time Win16 or Win32 programmer like Joel or me would know the term and be likely to use it in the context of Windows programming.

Keep in mind that Windows.h predated the modern definitions like uint32_t by many years. The original Windows developers had to come up with some names for these data types. It doesn't mean they weren't "real" programmers.

Although I suppose, given that floating point processors were far from being standard equipment at the time, you could argue that they were integer programmers...

overgard5• 6 years ago

Also, real programmers don't say DWORD.

Wait what? If you've done any windows coding in C it's practically lingua franca. I think the int{size}_t stuff is way better, but a "word" is common slang, and a double word is like a dword because nobody wants to write that much

Also, the "next and previous points xor'ed into a DWORD" hack mentioned at the end? This isn't duct tape programming. It's absurdly context-dependent premature optimization

Man I'm writing code for an arduino right now and that kinda optimization isn't rare for a machine with 2k of ram. It's not absurd it's useful to be able to do that. Sure don't do it on your 4ghz intel or whatever but techniques like that are useful.

kostyal• 6 years ago

I always liked this quote by Jamie Zawinski, from Coders at Work

"I know it’s kind of a cliché but it comes back to worse is better. If you spend the time to build the perfect framework…release 1.0 is going to take you three years to ship and your competitor is going to ship their 1.0 in six months and now you’re out of the game. You never shipped your 1.0 because someone else ate your lunch. Your competitor’s six-month 1.0 has crap code and they’re going to have to rewrite it in two years but, guess what: they can rewrite it because you don’t have a job anymore."

rimiform2• 6 years ago

Shouldn't that worry you, rather than excite you? He's just saying "worse is faster", not "worse is better". Especially that last sentence is kind of terrifying: "they can rewrite it because you don’t have a job anymore." Basically, everyone who takes the time to write good code will be outcompeted by bad code written quickly, until eventually everything is bad.

0x4d464d48• 6 years ago

I think 'annoy' is usually a more appropriate word than 'worry' (but not always). Markets and users are stochastic with no one really knowing what users are going to ask for, nor what competitors release and cause users to demand, well designed or not.

What a lot of these conversations boil down to is finding the sweet spot between over/under engineering which is subjective. Are we wasting time solving problems that dont matter or are we shipping with a 'just enough' type of mentality and risk exposing our users to buggy software?

No one likes to work with bad code but if its siloing itself into inconsequential CRUD apps or frivilous games/social media it's just annoying to experience/work with. Its when it creeps into systems or applications where consequences for failure are dire, think Lion Air crash in 2018, that Im worried.

capiki• 6 years ago

He's definitely saying "worse is faster" but he's also saying "worse is better," if what you're measuring is how good you're at shipping a product. And I think if you change "bad" to "good enough" (to be better than existing products) then you're right.

notriddle• 6 years ago

“Eventually?” Everything already is bad.

soco• 6 years ago

We're already giving all jobs to low cost centers and low cost employees and look where this is taking us: take home automation products as an example.

lmarcos• 6 years ago

Which is in some way sad.

I know we are not hired "to write code" but "to ship products", and that everything is done in "sprints" and we must be very agile in doing them because they're always non-stop, and quality comes second (unless, of course, you are really skilled and can ship high quality code in half the time it usually takes), and customers first and "we'll polish it in version 2" and...

I guess I feel "guilty" of writing good code just for the sake of writing good code (and taking time for doing so). At work code comes second, product comes first. I accept it and I try my best, but at the end of the day the code I write at my own pace at home for side projects, now that's the code I like to write.

dpc_pw• 6 years ago

It's easy to do things slowly, by the book, taking all the time in the world, while the world is waiting. It's part of the skill and mastery to know which corners to cut and which things are essential and which can be fixed later, and judge the effort to result ratio accurately.

twistedpair2• 6 years ago

It's not so binary though. If "perfect" takes 3 years, and "mad-max" takes 6 months, how good and maintainable a product can you muster in 9-12 months?

There is non-linearity here where you beat the "6 month" competitor the following year, because their product is buggy as hell and unmaintainable, but you keep cranking out solid features on your maintainable codebase/infra.

rimiform2• 6 years ago

I really feel as if the author should put the most important information right at the beginning of his post, namely that his coding philosophy is "you’re not here to write code; you’re here to ship products."

Of course the way to properly tackle these two objectives is going to be vastly different, and Joel just has the second objective, rather than the first. The problem is that almost everyone who complains about what you might call 'bad code' thinks that "writing good code" is the end goal, whereas almost everyone who writes said 'bad code' has "shipping products" as their end goal.

From the outside that might look like a difference in design philosophy, but really it's more a difference between someone who codes well because they care about their product and someone who codes badly because they have to put food on the table.

overgard5• 6 years ago

So one thing I think that gets lost is, to me, the most important quality in code is: how easy is it to debug. Like, I remember once having to maintain this guy's code where he just went crazy on C++ templates. And like, he proved that he's very smart and clever, but it made his code practically unusable since nobody could figure it out, and trying to debug a template masterpiece is like the hardest thing in c++ coding because the errors are impenetrable.

Coding isn't about proving your smartness it's about making something people can use and work with.

joatmon-snoo• 6 years ago

Forget about inherent complexity, just having good error messages is hard!

I spent hours this week debugging a test that kept failing in

  EXPECT_EQ(result.status(), SUCCESS)

Since this was a codebase I hadn't ever worked in before, I spent quite a while flailing around trying to figure out where I needed to add logging and eventually realized that `result` had more error details that the assertion was hiding from me.

I have a similar gripe with Python's unittest: because some of the assertions allow `expected` and `actual` to be interchangeable, there's always some extra mental overhead to remembering which is which. For example, compare this:

  >>> unittest.TestCase().assertEqual(1, 2)
  Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    File "/usr/lib/python3.9/unittest/case.py", line 831, in assertEqual
      assertion_func(first, second, msg=msg)
    File "/usr/lib/python3.9/unittest/case.py", line 824, in _baseAssertEqual
      raise self.failureException(msg)
  AssertionError: 1 != 2

with Java's Truth (https://truth.dev):

  assertThat(actual).isEqualTo(expected)

which results in errors like this:

  java.lang.AssertionError: expected:<[guava, dagger, truth, auto, caliper]> but was:<[dagger, auto, caliper, guava]>
    at org.junit.Assert.failNotEquals(Assert.java:835) <2 internal calls>
    at com.google.common.truth.example.DemoTest.testBuiltin(DemoTest.java:64) <19 internal calls>

sail0rm00n• 6 years ago

If only they had use VERIFY_OK/EXPECT_OK which prints out the error message :(

https://chromium.googlesource.com/chromium/src/+/master/net/...

superkuh2• 6 years ago

Coding isn't about proving your smartness it's about making something people can use and work with.

No, that's what business is about. Coding isn't just for business.

overgard5• 6 years ago

Realistically who is studying your code? Probably nobody unless you made something super famous. Otherwise youre what trying to show off? It's a lot of effort to get people to be: "neat, im confused" (Believe me I say that as a person that's written a lot of "im being clever" things to the delight of nobody)

zelphirkalt2• 6 years ago

Actually coding is about both. Proving your smartness _by_ making things people can work with, because they are simple. Creating a C++ template madness is not that smart in the end.

Really smart people invest time into making their code very readable and at the same time manage to give it the required layers of abstraction and flexibility in a way that does not confuse the reader. They make their code in a way, that enables them or others to write tests for it. They think about the concepts needed in the project. Find a good trade-off between simplicity and flexibility and if you are a true master of finding good abstractions, manage to choose a simple abstraction to get simplicity and flexibility at the same time. At least that is being smart to me.

overgard5• 6 years ago

"Proving" you're smart is a terrible idea, if you're smart it'll come across pretty quick and if it's not obvious to other people the attempt seems pretentious.

I don't think you should guard smartness by pretending its the same thing as practicalness. I know a lot of people that spent time on brilliant work that had no point. They were smart as hell just impractical

If I'm hiring a coder and I get one that writes the boringist work, or a brilliant person that writes their code as a quine, I respect quine guy but I want boring guy.

coip• 6 years ago

Elegant simplicity is much like the adage in racing: slow is smooth, and smooth is fast

twistedpair2• 6 years ago

This has been going on for over a thousand years. Read Ptolemy's book from the 2nd century to describe the solar system. It used hundreds of "epicycles" to make everything line up. Then Newton comes up with three tiny equations that describe the solar system's motion even better in the 18th century.

As they say, "anyone can design a complicated system", but if you're really smart, you can make a simple system. It's no different in programming.

Uhhrrr3• 6 years ago

jwz's response is worth reading as well: https://www.jwz.org/blog/2009/09/that-duct-tape-silliness/

"It's such a strange article, in that it's mostly favorable to my point of view but with such a breathless amazement to it, like he's just discovered an actual unicorn or something. "Look, everybody! Here's a hacker who actually accomplished things and yet he doesn't fetishize the latest fads that I and all of my friends make our living writing about!" There's this tone to the thing like he just can't imagine that someone like me can exist."

dilyevsky• 6 years ago

Lol that ddos banner. Hey @jwz, you should look into serverless or something

Uhhrrr3• 6 years ago

Ha! Well, just cut-paste the URL instead of clicking, folks.

russianbandit• 6 years ago

... otherwise they’d have to admit that they just don’t feel smart enough to use what would otherwise be a perfectly good programming technique FOR SPOCK.

Is how I feel about Haskell.

The Duct Tape Programmer (2009)

Related Stories