A good developer has a natural, almost visceral aversion to complexity

http://sigpwned.com/content/complication-what-happens-when-you-try-solve-problem-you-dont-understand

by signa11 • 14 years ago

215 111 14 years ago

stcredzero6• 14 years ago

I've also met bright people, mostly coders, who have told me they adore complexity. Even one young man who advocated hyper compact self-modifying code.

rhizome4• 14 years ago

Last summer I was fired from a startup whose codebase was written entirely by miscreants like those. It was like a CS hazing experiment.

fleitz• 14 years ago

Let me guess... they were addressing hypothetical scaling issues for a site with < 100 users? Instead of optimizing the conversion rate on their homepage and mailers?

stcredzero6• 14 years ago

It was like a CS hazing experiment.

An old hand programmer once told me he could figure out whatever social/psychological pathologies from the code of the programming shop that wrote it.

mhd6• 14 years ago

Don't we all do that to some degree? There isn't a fixed drop off point for "too much complexity" that's fit for everyone. Consider programming languages, are APL or Prolog "wrong"? Or, compared to something like Oberon, are Ruby, Python and Java?

stcredzero6• 14 years ago

Don't we all do that to some degree?

Lots of musicians go through a stage where they are trying to see what they are capable of and make music as involved/complicated/ornate as possible. Many of them mature into musicians who try to find a few elements that work together.

gruseom3• 14 years ago

You're implying that APL is complicated. It isn't. It was designed for regularity and simplicity. Iverson and his group worked on that for 7 or 8 years (can you imagine!) before they began implementing it, and what they came up with remains one of the great high water marks of software design.

This brings out a problem with the OP and the discussion. We readily dismiss things as complicated that are merely unfamiliar. The converse is also true: we don't see complexity once we've become habituated to it. And we become habituated surprisingly quickly.

So how do we distinguish between complexity and unfamiliarity? Without some reliable distinction, this is all in the eye of the beholder.

rhizome4• 14 years ago

"Hit by a bus" contingencies are always an easy challenge to bring up whenever this mindset arises.

Produce2• 14 years ago

After five years in the industry, I'm mortally afraid every time I see one of those four wheeled demons speeding past me on the road.

ericbb• 14 years ago

Yes, I was thinking that too. I like simplicity and I like complexity.

I was also reminded of an old a comment from Guy Steele: http://www.youtube.com/watch?v=4LG-RtcSYUQ&t=1h21m11s

diminish• 14 years ago

It can be argued that, at no point we understand the problem fully and complexity is natural in any software after some development progresses.

bostonvaulter22• 14 years ago

Anyone have recommendations about the linked book?

Refactoring: Improving the Design of Existing Code http://www.amazon.com/Refactoring-Improving-Design-Existing-...

marcusf• 14 years ago

It's one of the classics of our trade; Even if you know everything there is to know about refactoring you should really read it just to have read it. So consider that my recommendation :)

sigpwned2• 14 years ago

That book changed the way I look at programming. For me, it was the first book I found that actually answered questions that I didn't have the vocabulary to ask yet, rather than just talking around them.

Disclaimer: I'm the author of the post, so that's not actually a different recommendation, just a more elaborate one. :)

greenyoda• 14 years ago

Another book on refactoring that I'd highly recommend is "Working Effectively with Legacy Code" by Michael Feathers. This book gives you lots of practical hints about how to start refactoring a large, messy code-base. While the task may seem overwhelming at first, you can start by getting small chunks of the code refactored and testable.

bostonvaulter22• 14 years ago

Wow, you got me there. "Working Effectively with Legacy Code" would be even more directly in-line with what I'm looking for. Way too much Java EE stuff in the code base I'm working with right now.

rhizome4• 14 years ago

Refuctoring: http://www.waterfall2006.com/Refuctoring.pdf

MaysonL• 14 years ago

A good developer has a natural, almost visceral aversion to unnecessary complexity.

There, fixed that for you.

'Make it as simple as possible, but not simpler' (Einstein, as quoted by Niklaus Wirth.

jfoutz• 14 years ago

quoth the article: "It’s only by diligently trying to avoid all complexity that one can in fact avoid unnecessary complexity."

Amusingly you seem to support the author's argument that running around making changes when you don't really understand whats going on leads to lower quality code.

quackingduck• 14 years ago

Well played sir

sigpwned2• 14 years ago

Author here.

I actually use that exact Einstein quotation in the article, attributed to the big man himself. One of my favorites.

thezilch• 14 years ago

Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.

-- Brian Kernighan of "K&R C" and AWK fame

23david• 14 years ago

Hm so that explains why it's so hard to debug rails apps. :-)

getsat• 14 years ago

If you're having serious issues debugging Rails apps (or any app built on a major web framework), you are doing things horribly wrong.

If you're just quasi-trolling/shitposting, please go back to Reddit.

cnbeuiwx• 14 years ago

Python has a tendency to create non-complex programs in my experience. Today I rewrote a shell script with Python and now its much, much more elegant and easy to understand.

I guess what Im saying is that the programmer need some help from the language to make the software non-complex. Even trivial things often becomes complex in low level languages.

boyter• 14 years ago

Agreed, till you hit a problem that requires 10,000+ lines of code. I love Python to bits but for this one large website I am working on I wish it was written in C# Java or some other statically typed language.

When you have a large project I will always take the compile time checking that tends to resolve a lot of bugs over unit tests to verify the same thing. Especially when it comes to large re-factoring efforts.

makecheck2• 14 years ago

One of the most important things programmers can learn is that you don't have to choose one language per project. Binding, say, Python and C++ in the same program can be very valuable (though it is not trivial, even with SWIG). Some code benefits from the simplicity, clarity, testability and huge standard library of the scripting language, and the remaining code gains performance and expressiveness to handle its greater complexity.

thespin6• 14 years ago

By this measure, there are a lot of developers who are not good.

So what are they? Great?

Seriously, how do we describe developers who have a visceral perversion toward complexity?

There are a lot of developers who love complexity.

balloot3• 14 years ago

My description would be "bad developers".

xsmasher• 14 years ago

They are arsonists masquerading as firemen.

njharman• 14 years ago

Inexperienced. That is based on my own 20yrs of experience watching (some) developers mature.

mhd6• 14 years ago

"There are a lot of developers who love complexity."

Regular expression complexity or AbstractSingletonFactoryImplementationBeanStoredProcedure complexity?

balloot3• 14 years ago

Both are awful. I have a particular disdain for regex abuse. I like to say that there are 3 stages in the use of regular expressions.

1) You don't get them and think they're magical

2) You learn regular expressions, and find all kinds of places to apply your knowledge of greedy modifiers and positive lookbehinds.

3) You learn enough to realize that regular expressions tend to be extremely brittle and try as hard as you can to avoid them.

There are a lot of very dangerous developers stuck in level 2.

thespin6• 14 years ago

The hero is the negative coder. McIllroy said something to that effect. The hero is the simplifier.

zeekay• 14 years ago

Speaking of McIllroy, reminded of this relevant story: http://www.leancrew.com/all-this/2011/12/more-shell-less-egg....

thespin6• 14 years ago

Wasn't there also a book called "Programming Pearls" by Jon Bentley?

fsckin• 14 years ago

A similar nugget of wisdom:

"One of my most productive days was throwing away 1000 lines of code." -Ken Thompson

thespin6• 14 years ago

I'll bet he was referring to code he wrote himself. It's slightly more complicated (pun intended) when it's someone else's code you are negating.

thespin6• 14 years ago

I'll bet some developers are literally "embarrassed" to write and share "simple" code.

Maybe "trivial" is good.

If you can "write something in a weekend", it's easy to maintain. If you lose the code, you can rewrite it. And maybe it will be better the second time.

When a project becomes so complex you cannot easily rewrite it, is that a good thing?

kijin• 14 years ago

Well, we know what happens when something becomes "too big to fail", don't we?

balloot3• 14 years ago

Best and most correct headline I've read on HN in a long time. Well done.

rmATinnovafy• 14 years ago

Once I got over the fact that simple code does not equal bad code was I able to level up in skills.

Now I focus on simple solutions. That is why I've come to enjoy Ruby.

ww5202• 14 years ago

While it's ONE of the nice principles to remember; however, it's not the golden rule. Often times it's a matter of tradeoff. The complexity upfront can be for the simplification later down the road. E.g. defining a DSL would be an extra complexity upfront but simplifies and streamlines work down the road. A good developer knows how the make the judgement call to balance the tradeoff in complexity.

oskarth• 14 years ago

That's not what I'd call a tradeoff. Problem complexity isn't the same thing as incidental complexity. If a DSL is useful for the problem domain, it's not "extra complexity", it's a simple (albeit hard to implement, simple != easy) and elegant solution.

ww5202• 14 years ago

A DSL is an additional functionality adding more components to the system and thus increases its complexity. A DSL incurs extra learning cost that new developers coming onboard need to spend more effort in learning.

You can have a simple library or util module with the standard function API instead of building a DSL. They work just as well in many designs.

It is a tradeoff when deciding to use DSL or not.

Daniel_Newby• 14 years ago

I'd say that a good developer has had his natural affection for complexity beaten out of him by that merciless bitch experience.

BlackJack• 14 years ago

Beauty is more important in computing than anywhere else in technology because software is so complicated. Beauty is the ultimate defence against complexity.

— David Gelernter

This resonates with me. The article makes a fine point about removing unnecessary complexity, but in the real world you can't be perfect in that regard - not for a large system anyways. So maybe we should allow a little more complex code if we can use the beauty of the product to hide it.

chriszf• 14 years ago

You can't be perfect but you can at least try to reduce complexity. Saying "just paint over it with pretty colors when you're done" invites unchecked complication. That attitude is what drove all the monkey/ninja patching that we saw with some early Ruby modules. They were beautiful, yes, but it made some code impossible to reason about.

lobo_tuerto• 14 years ago

What I take from this is we should strive for the most elegant solution to a particular problem. Be it algorithm or software architecture.

For example, if I had in some way, the catalog of all the possible solutions to problem X ready in my mind, I'd pick the most elegant one. Not necessarily the shortest, but the most elegant, the one where you can spot the _solution's outlines_ easier.

I think seeing and appreciating this outlines would make you a connoisseur of fine software. You would strive for beauty and elegance, and appart yourself from unnecessary complexity.

swernoxidation2• 14 years ago

"It’s always bothered me that you can simplify “Simplify, simplify, simplify” ... <facepalm>

makecheck2• 14 years ago

There's a tricky problem that comes up in teams. A "bad" developer can hack and slash and appear to implement new features in a certain amount of time, and a "good" developer who takes the time to do everything right may actually produce less of obvious value in the short term. When approaching software restructuring, you have to make damn sure that your management chain understands the importance of the refactoring and can evaluate the time you spent for its actual worth. If your organization uses "metrics" or other weasel words to examine productivity, tread carefully.

It's also worth noting that even if you believe you know how to improve/simplify/whatever, it may be worse than you think and sometimes you don't figure out the extent of the complexity until days have gone by.

This, like many things, is a trade-off and not a set rule. In brand new code, by all means make it small and pretty and documented and whatever else would win it an award; but in established code bases be smart about what you "fix".

gbog2• 14 years ago

In three paragraphs of readable language without any rhetorical tricks you dug much deeper than the OP, thanks.

martininmelb• 14 years ago

I'll try to summarise in two words: "Technical Debt".

drumdance• 14 years ago

At my last company we had a co-founder like this. He would implement stuff extremely quickly, often at odd hours to fix a pressing bug. However, the code was just garbage from a maintenance standpoint.

It was hard to criticize, though, because he pretty much single-handedly got the company off the ground by quickly implementing features people were willing to pay for. This code was good enough to get the company up to $2 million annually.

The downside is we spent about a year re-writing the software to get it to a state where the company could scale to higher levels. During that year we had to stop adding new features.

I remember getting very frustrated with the code quality, but I often wondered whether that was just a price we had to pay to get cash flow positive.

dennisgorelik2• 14 years ago

Yeap, that was the price. It's called technical debt and sometimes it makes sense to have it.

Dirty code not always has to be refactored. Most of dirty code just dies together with useless functionality (wrong guess on business side).

sliverstorm• 14 years ago

What is it called when a certain amount of complexity is introduced, that serves to reduce overall complexity?

rhizome4• 14 years ago

Example?

gbog2• 14 years ago

Refactoring. You have to move this logic from A to B, thus adding some complexity to B, reducing the overall complexity because this logic really belongs to B.

Note that during refactoring you will have twice the logic, which is the worst state, so you can't stop in the middle and say you'll finish later.

tikhonj• 14 years ago

The problem with these sentiments is that complexity is relative--what one developer finds simple another can completely reasonably find complex. So yes, excess complexity is bad, but what defines excess? Are two Python nested for-loops significantly "more complex" than, say, foldM? I think so, but others obviously disagree--van Rossum thinks even normal folds are more complex!

The real interesting question is not whether you should strive to make your code complex (clearly not) but rather what "less complex" actually entails. Does this mean less moving parts? More parts that are simpler? More verbose code that is more explicit? Less verbose code that is easier to read at a glance? More math? Less math? Code that looks uniform? Code specialized to the given domain (and therefore less uniform)?

I have strong opinions on these questions, but I have no answers. I've met others with strong opinions exactly the opposite of mine. They don't have the answers either. It is, ultimately, not an easy question.

spacemanaki• 14 years ago

An interesting assertion in Rich Hickey's Simple Made Easy talk is that simple and complex are almost objective qualities, at least when you accept the definitions he puts forth (simple: one fold/braid with no interleaving). I think he's right, but I'm not sure I agree entirely, only because I haven't spent enough time thinking about it and trying to apply the ideas to real world problems. I do think that most programmers can call out complexity without a lot of disagreement. What you describe regarding fold vs for-loops touches on his definition of easy, that is to say "close at hand". Fold is easy for a functional programmer, for-loops for a Python programmer. Their simpleness might be a different matter.

http://www.infoq.com/presentations/Simple-Made-Easy

kalininalex5• 14 years ago

The answer actually IS objective. The limit of complexity is defined by the fact that our short-term memory can simultaneously hold only five entities, plus or minus two. So, if the code requires one to hold more than seven elements in mind simultaneously - no one will be comprehend it, even the programmer who wrote it.

I once inherited a module written by a more junior developer. He indiscriminately used several dozen global variables that were used and modified in various parts of the program without much of a system. I spent about two months trying to understand how the module worked without much success. Ultimately, I had to completely re-write it (with great difficulty) to get rid of global states, and bring the number of entities affected by each function to under five-to-sever.

This biological limitation affects our development tools. That's why we have object-oriented programming. The goal is to keep the number of items we have to deal with at any moment under that magical limit of five to seven. Incapsulation is a key component of OOP used for this purpose. Without it all will be lost.

Sometimes I think of an alien race capable of keeping track of 100 items simultaneously, and try to imagine what their programming languages look like.. :)

dennisgorelik2• 14 years ago

That limitation is not exactly biological. I'm pretty sure that alien mind (or artificial mind) would have pretty similar limitation. It could be "10 entities" instead of "5 entities", but the number of simultaneous entities in "processing model" would not be much higher.

The real reason for such limitation is that number of possible connections between objects is growing very quickly.

For example, 2 entities connect with each other by 1 connection.

3 entities connect by 3 connections.

4 entities connect by (4 * (4-1)) / 2 = 6 connections

5 entities -- 10 connections.

7 entities -- 21 connections.

10 entities -- 45 connections (4.5 times higher than for 5 entities).

Number of possible connections between entities defines complexity of the model.

grncdr2• 14 years ago

The question that comes to mind for me: how do we define "entities" in the context of writing software?

Especially given gorelik's observation that complexity really derives from the combinatorial explosion of connections, are objects with internal state the best way to manage that complexity? If every call of a public method from another object is a connection in the graph our entities are an (object, message) pair, and a situation involving 3 objects with 3 public methods each exceeds our limitations.

Contrast this with a pure-functional approach. If you can express your algorithm with immutable data and operations that create new transformations of that data, you could (hypothetically) have fewer things in mind at any one time: the data being operated on, and the single operation being performed. It doesn't break down quite so cleanly in reality, but use of pure functions where possible largely obviates the benefits of state encapsulation as provided by most modern OO languages.

batista5• 14 years ago

From the post: """It’s always bothered me that you can simplify “Simplify, simplify, simplify”"""

No, you can't really. You can just say "simplify" but it does not convey the same message.

That is, it conveys a suggestion to simplify (the first part) but it doesn't convey that simplification has so extreme importance that it's worth to repeat the message (the second part).

kami88453• 14 years ago

Simplify * 3 ?

batista5• 14 years ago

Not the same because it only makes sense to programmers, not a very large percentage of Thoreau's readership (or today's general population).

zaptheimpaler• 14 years ago

Congratulations! You just won the pedantic comment of the day award!

swernoxidation2• 14 years ago

Thoreau just might have been ironic.

batista5• 14 years ago

Yes, and both the poster's thought and my response might have been tongue in cheek too.

It's not like I thought of it as a serious question, I just added it for the benefit of anyone confused by the contradiction.

jQueryIsAwesome• 14 years ago

"Simplify!!!"

Ygor• 14 years ago

In the paper No Silver Bullet [1], Brooks makes a distinction between accidental complexity and essential complexity. He claims that most complexity remaining in contemporary systems is essential.

Moseley and Marks disagree with his premise in the paper Out of the Tarpit [2]. They propose an approach based on functional and relation model to minimize accidental complexity.

Even if you don't agree with the conclusions or some of the premises in this papers, both provide an excellent read on the subject.

[1] http://www.cs.nott.ac.uk/~cah/G51ISS/Documents/NoSilverBulle...

[2] http://web.mac.com/ben_moseley/frp/paper-v1_01.pdf

spenrose• 14 years ago

I got about halfway through Out of the Tarpit. I found their claims implausible. The way to refute my reaction is to build a great system using their approach. Anyone done it?

jasonlotito• 14 years ago

YG WQ J SABVB 7UH TYHGFAW CDR

eagsalazar• 14 years ago

The worst feeling is working in someone else's overly complicated, messy code. It's like having to work covered in shit. The ick feeling is constant and overpowering.

Produce2• 14 years ago

I think that a good developer is one who can strike a balance between correctness (solving the problem at hand), elegance (the system is no more or less complex than it must be) and consistency (the "culture" of the program is the same in all of it's components).

I would put an aversion to complexity in the same box as an attraction to it. I.e. atheists are fundamental in the same way that christians are. Agnostics know what's up.

jiggy2011• 14 years ago

Something I have started doing; Whenever I create anything, whether it is a program , website copy and email or whatever is take a step back and say "How can I make there be less of this?" rather than "how can I make this better" or worse "cleverer".

I think education in general rewards complexity, getting bonuses in English class for using long passage of flowery words with many synonyms or extra credit in CS for adding 200 lines but reducing O(n^2) to O(n log n).

meric• 14 years ago

>> getting bonuses in English class for using long passage of flowery words

That depends on who your teacher is. My teacher taught that good english should be understood by as many people as possible while retaining all of its meaning. The goal of English is to communicate.

He was a good teacher because the top 65% of our class got into the top 35% of the state for English, while our school was below average in just about every other subject.

6ren• 14 years ago

a related formalism is http://en.wikipedia.org/wiki/Kolmogorov_complexity, the idea that the shortest possible program defines the complexity of the problem. Thoughts:

- you might not have seen enough cases be able to characterise the true problem, e.g. in some situation a constraint exists, but you've only seen one instance of that situation

- reusing standardized components can enable you to solve the problem more quickly (even though one-size-fits-all doesn't really fit)

- Kolmogorov complexity doesn't account for efficiency, space nor time, only that the eventual answer is correct. Efficiency often requires ugly hacks

cma• 14 years ago

There is a resource bounded version of Kolmogorov complexity. It might even be a more useful measure for certain things. For example, a simple enumeration can prove any arithmetical statement that can be proved, but it can take time exponential in the size of the solution. With some resource weightings you might get a better idea of how hard the problem is in actual practice.

RyanMcGreal• 14 years ago

> I may be an introvert.

Introvert != Misanthrope.

A good developer has a natural, almost visceral aversion to complexity

Related Stories