Some lesser-known truths about programming

http://dotmac.rationalmind.net/2010/08/some-lesser-known-truths-about-programming/

by AndrewDucker • 16 years ago

118 55 16 years ago

dotmac.rationalmind.net

wccrawford• 16 years ago

The first few, I don't fully agree with. I think the numbers are exagerated. But these 2 have more truth:

"Although most software is made by teams, it is not a democratic activity. Usually, just one person is responsible for the design, and the rest of the team fills in the details."

When things are designed by 1 person (IME) they work better and have a more flowing work flow. Designing in a committee has been a freaking nightmare.

"Programming is hard work. It’s an intense mental activity. Good programmers think about their work 24/7. They write their most important code in the shower and in their dreams. Because the most important work is done away from a keyboard, software projects cannot be accelerated by spending more time in the office or adding more people to a project."

While I don't 'code in the shower', I do think about my projects constantly. An 'ah hah!' moment can come at any time. Just the other day I left for lunch and in the car on the way I thought of a possible reason for an issue we were experiencing. I was getting nothing while staring at it.

kranner• 16 years ago

I agree with you - I believe that, in most cases, the whole industry suffers from too many cooks spoiling the broth.

cagey• 16 years ago

"When things are designed by 1 person (IME) they work better and have a more flowing work flow. Designing in a committee has been a freaking nightmare."

I completely agree. Yet in industry, being that one person (like I am) leads to great management consternation to the tune of "what if you're hit by the proverbial bus?" which leads to "you need an understudy", which leads to expending 2x the resources to achieve <2x results (the time spent selecting, training and guiding an understudy is definitely nowhere near 0, even when the choice of understudy turns out to have been a good one), and there can't help but be some feeling of "training your replacement". Industry loves interchangeable parts (people), and a one-man-band (domain expert or similar) grates against that ethos.

mseebach• 16 years ago

I hate the proverbial bus. It seems to assume that central employees should be completely dispensible.

If a key employee, being the architect or even the CEO, is hit by a bus, the company will take a hit. If employees are a central, strategic assets, you can't go around killing them off and expecting no impact on your business. Deal with it.

The important metric is whether the company/project can recover, if it is resilient. Just because one guy drives the decisions, it doesn't mean that the team filling in the details don't know the code and the motivations behind it, and there can't be a guy from that team who could take a promotion, or that the team couldn't be reasonably productive for a while, while a new architect is brought on.

skybrian• 16 years ago

That's a bad way to think about it. You probably don't want to work on the same project forever anyway, and if you're so smart, teaching other people what you know is the most important thing you should be doing.

The bottleneck on many projects (especially those that are either poorly defined or particularly novel) is not coding speed, but rather learning speed. The faster the team learns about the problem it's really solving, the environment the solution has to live in, the people who will have to use the software, and what the possible solutions are, the faster you're done.

hga• 16 years ago

Apropos of the 1 person design thesis, the Antipatterns book said that only 1 in 5 programmers "get" abstraction. That's why democratic design processes tend to fail: the ones who get it are outvoted by the ones who don't.

And that certainly matched my quarter century experience in the field, and boy was it frustrating to watch teams try to build frankly impossible systems.

Heck, I'd be happy if more people got and could use big O notation. In one particularly unhappy case my peers just couldn't grok that the design would vastly outstrip the database's maximum transaction rate in normal customer configurations (number of client workstations).

makmanalp• 16 years ago

That's like saying only 1 in n people have food, shelter and education. The ones that do have this property are likely to be with their peers (or close), as are the ones that don't.

dimitar• 16 years ago

What makes programming so unique?

Why cannot we generalize?

I think most of the same conclusions apply to teachers, lawyers, salesmen, marketers, architects, physicians, nurses, engineers and scientists, whatever.

Wherever knowledge is valued in a job and there are intricacies lots of research, smart planning and thoughtful execution with a great commitment is going to make the worker a lot more productive.

j_baker2• 16 years ago

"What makes programming so unique?"

Programming isn't necessarily unique any more than any of the fields you mention are unique.

"Why cannot we generalize?"

... but here's where you're stretching it a bit. I agree with you that programming isn't as unique as a lot of these blog posts make it out to be. But there's no ignoring the fact that different fields have different requirements.

For example, I don't think that a good programmer necessarily has the skills to be a good teacher (unless it's a computer science teacher, and even then it's debatable). Computer science involves translating logic into code. Teaching requires you to make logic suitable for people less educated than yourself to understand. That's not to say that a good programmer couldn't learn to teach or vice versa. It's just that they're orthogonal skill sets.

jamesbritt• 16 years ago

"That's not to say that a good programmer couldn't learn to teach or vice versa. It's just that they're orthogonal skill sets."

In college (for my CS degree) I had to take a technical writing class. I loved it. I knew next to nothing about the field when I started, but soon found the idea of translating complex ideas and concepts into more easily understood formats quite intriguing.

It was, in some ways, much like programming. I came to appreciate how hard it is for some people to understand some ideas (a point later reinforced when I started tutoring), and learned ways to improve descriptions and presentation.

It seemed to me that if you are involved in some geeky tech field it behooves you to learn how to communicate with Earthlings. Bizarrely, when it came time to pick a minor, my counselor made a comment to the effect that with a CS major I should be looking at, oh, a EE or ME minor sort of thing. Not something out in left field, such as tech writing. I hadn't yet told him that in fact that was the minor I was going to pursue. :)

Well worth the time and effort; if you have great ideas but cannot adequately express them, you're losing.

powrtoch• 16 years ago

People can say things like "lots of research, smart planning and thoughtful execution with a great commitment", and believe in it as a general principle, without applying it or even knowing how to apply it to themselves. A lot of these do translate in some way to more general rules, but I still think it's helpful to have targeted insights. "Planning is good. For you this means think before you code, make apps easy to maintain and scale, avoid hacks, etc." Without that a lot of people will fail to notice the connection to their specific activities.

yummyfajitas2• 16 years ago

Programming isn't unique in the set you list, but it is unusual.

Engineer, scientist and programmers are probably in an unusual situation - they are exploring a primarily unknown solution space.

Most jobs live in the well explored, primarily understood areas of solution space, and consist of simply applying already known solutions repeatedly. Teachers, doctors (mostly) and most nurses just perform a set of rote tasks of varying complexity. The best way to improve performance in these fields is to write a script/checklist/decision tree and order everyone to follow it.

(In the case of doctors, I exclude a few at the very high end. Most doctors are not House.)

jacquesm• 16 years ago

I'm with you on the scientist and the programmer, but the whole purpose of engineering is to work explicitly within the known solution space to be able to guarantee the outcome of their work.

Engineering is applying the lessons of science to achieve predictable results.

rlpb• 16 years ago

> Engineer, scientist and programmers are probably in an unusual situation - they are exploring a primarily unknown solution space.

This depends on what you are doing. Some programmers are exploring a primarily unknown solution space, and the best programmers probably self-select as this is difficult.

But the vast majority of programmers are busy applying the same well understood techniques to different problem domains.

atlei• 16 years ago

"What makes programming so unique?"

E.W.Dijkstra, "The humble programmer", ACM Turing Lecture 1972

"Let me conclude. Automatic computers have now been with us for a quarter of a century. They have had a great impact on our society in their capacity of tools, but in that capacity their influence will be but a ripple on the surface of our culture, compared with the much more profound influence they will have in their capacity of intellectual challenge without precedent in the cultural history of mankind. Hierarchical systems seem to have the property that something considered as an undivided entity on one level, is considered as a composite object on the next lower level of greater detail; as a result the natural grain of space or time that is applicable at each level decreases by an order of magnitude when we shift our attention from one level to the next lower one. We understand walls in terms of bricks, bricks in terms of crystals, crystals in terms of molecules etc. As a result the number of levels that can be distinguished meaningfully in a hierarchical system is kind of proportional to the logarithm of the ratio between the largest and the smallest grain, and therefore, unless this ratio is very large, we cannot expect many levels. In computer programming our basic building block has an associated time grain of less than a microsecond, but our program may take hours of computation time. I do not know of any other technology covering a ratio of 1010 or more: the computer, by virtue of its fantastic speed, seems to be the first to provide us with an environment where highly hierarchical artefacts are both possible and necessary. This challenge, viz. the confrontation with the programming task, is so unique that this novel experience can teach us a lot about ourselves. It should deepen our understanding of the processes of design and creation, it should give us better control over the task of organizing our thoughts. If it did not do so, to my taste we should not deserve the computer at all!"

Edit: http://userweb.cs.utexas.edu/~EWD/transcriptions/EWD03xx/EWD...

extension3• 16 years ago

Programming is the first discipline that is not subject to certain fundamental limitations intrinsic to many other fields of work and, as a result, has revealed new fundamental limitations that nobody has ever encountered before.

palish3• 16 years ago

It’s easier to throw away bad code and start over than to change it.

You had me nodding along until I came across this. This single sentence is not just wrong, it's so wrong that it unfortunately calls the rest of the article into question.

hnal943• 16 years ago

It is easier. Whether it's better or faster is another question.

palish3• 16 years ago

The whole reason to even list any programming truths is because they would make you a better and faster programmer.

extension3• 16 years ago

Depending on how much functionality needs to be changed, and how the new developers compare in skill to the old ones, it could very well be easier and a better choice in every way.

jemfinch• 16 years ago

Studies have shown (see McConnell's *Code Complete for citations) that bugs are far new likely to recur in previously buggy code than in code not yet found to be buggy. Since so much time in development is spent in testing and debugging, it's not at all unreasonable (based on studies or on my personal experience) to claim that it's easier or better or faster to rewrite code rather than try to fix it.

This dovetails quite reasonably with Brooks' claim that we should plan to throw one away, because we will anyway.

j_baker2• 16 years ago

You know, I'm always skeptical about any article that refers to "truths". Why is it that we are so intent on there being just one "true" way to be a good programmer? Can't we accept that there are multiple paths to becoming good?

stingraycharles• 16 years ago

Not only are there multiple paths to becoming good, but "good" depends entirely on the requirements. In a start-up, a good programmer often is a programmer that has the ability to complete requirements as fast as possible. You don't want a start-up developer writing 10 lines of code a day on average while being on the hunt for the "best" solution, if there even exists such a thing.

It all totally depends on context, and these articles always use the context from which the author is biased.

Juha• 16 years ago

I somewhat disagree with the point that good programmers think about the problems 24/7. Even good programmers loose motivation if there is bad company culture or some other de-motivating factors. I find that true motivation and true interest for the project are very important in software development but so many companies neglect it.

thingie• 16 years ago

Well, 24/7. That's simply too much. I had (well, still have) real-life problems that I not even closely think about 24 hours a day, all the week, for the last half year, and I don't get better in solving it. No. I'm slowly getting mad.

alexkearns7• 16 years ago

Not so much lesser-known truths as absolute bullshit.

The author describes developers who spend much of their time writing code as "lazy, ignorant and arrogant". I think it is the author who is being lazy, ignorant and arrogant in thinking that he knows the truth about programming.

And the idea that a developer will only write a handful of lines of code each day that will end up in the final product might be true for a developer working on an established piece of software but anyone who has worked for a start-up or on their own software will just laugh at the idea of writing just 12 useful lines of code a day. A good developer can easily do hundreds, if not low thousands of lines of code in a day if pushed, and this code does often end up in the final product.

Plus, he seems to think that a great programmer is someone who connects together bits of code created by others, rather than writing his own code. That's ridiculous - is someone who uses another person's code a better programmer than the person who wrote the code he is using, in which case I am a much better programmer than John Resig, say, because I simply use jQuery rather than having written it myself.

I think this article is a perfect demonstration of the Dunning-Kruger effect. Sorry for being so blunt but the article bloody annoyed me.

Retric3• 16 years ago

This is looking at the average day of the average programmer over the lifetime of the average product. Working on a 6 month old project it's vary easy to produce negative lines of code as you refractor. But, when you look at the average there is little reason to count time spent creating code that was soon replaced as useful.

It's also easy to think you are vary produce when you produce 10k lines of code in the first six weeks of a project, but the reality is over time most of that is going away. And writing more than 10 lines of code a day that sticks around for more than 2 years without making people wanting to replace it is hard.

PS: Google, for example, often scraps the source code of any company it acquires so the real metric of lines of code produced per day over the first six months is often zero in a successful project. Many a startup has failed and nothing of value was lost.

alexkearns7• 16 years ago

I think maybe you work in a completely different field of programming to me. Most lines of code I have written in my career have ended up in the final product. The same is true of most the developers I know.

nocipher• 16 years ago

There is some truth in the article. Coding is easy. It's all the other aspects of software development that are difficult. I know that the biggest hurdle for any programming project I've ever been involved in is deciding what will be the different parts of the system and then how those parts will interact. Once I start writing code, I can program very quickly because I don't have to make hard decisions.

Even so, a few thousand lines of a code a day is a bit much for anything non-trivial. To write that much code in a single day means that person must have done little else besides coding. Unless there was a period of about a week of design before that kind of coding, I'd expect the code to be rather kludgy because everything would be decided immediately. A few thousand lines of code, though, is definitely not something I'd consider average.

A /small/ project that added a new feature to an existing system took about 30 hours for the initial version over a two week time period and about 800 lines of code. Granted, I don't know exactly where my programming abilities lie skill-wise, but I can't imagine someone being 30x or 40x more productive than that as you suggest.

And maybe I'm just new around here, but if was going to argue for the Dunning-Kruger effect, I'd begin by arguing that I am clearly not suffering from the effect myself.

alexkearns7• 16 years ago

We all suffer from the Dunning-Kruger effect, to varying degrees. I would never dream of arguing I don't suffer from it. I though do not go around claiming I know the truth about programming.

I think that perhaps there are two types of programmer - those who like to spend most of the time researching, drawing diagrams and the such like, and the others who just get on with it.

The suggestion of this article is that those who spend most of their time planning, rather than programming are somehow superior - my experience with developers has been the exact opposite.

ntoshev• 16 years ago

Don't worry about the lines of code you write, I'm pretty sure even Dr Norvig can't write 50 of these per day: http://norvig.com/spell-correct.html

cubicle67• 16 years ago

to play devil's advocate: could it be that your comment is a perfect demonstration of the Dunning-Kruger effect?

loup-vaillant5• 16 years ago

I think it's even simpler: he took it personally. He spends much of his time writing code (whatever that means), so he thinks the article is calling him lazy and ignorant and arrogant. (That's not even true, by the way. The article says "lazy or ignorant or arrogant".)

Anyway, it triggered a classic defence mechanism: "what is calling me bad must be wrong". So he attacked the part of the article that offended him. In the process, he overshoot, and tried to discredit the whole article (by saying "complete bullshit").

(By the way, I only talk about the emotional process that likely triggered alexkearns' comment. That doesn't make it wrong. Nor right.)

tomlin• 16 years ago

Something funny about the subject and this site giving me a "Error establishing a database connection" error.

Typhon• 16 years ago

There's nothing wrong with this article in my opinion, except for the title.

« Some lesser-known truths about programming » sounds better than « A few things that those of you who never read Dilbert don't know about programming with a team for a big company », i guess, but the latter is much more accurate.

diego_moita• 16 years ago

Actually, most of these truths should be very well known; I've seen them thoroughly explained and documented in books like 'Code Complete', 'Peopleware', 'The Mythical Man-Month', 'Refactoring', etc.

If there are people thinking they're bullshit, then the author is twice right.

ww520• 16 years ago

A dentist friend once asked me why is software development so difficult and cost so much. I told him that's because a competent developer has to become an expert in the domain of the application in addition being an expert in programming. It's like learning a new field every time you develop a new application.

vog2• 16 years ago

This is so true.

Often, people try to circumvent that problem by providing complete specs upfront. But those are doomed if they lack the old problem, i.e. if they were written by people who are missing expertise in either programming or in the domain.

Another way to circumvent the problem is to establish a good communication flow between the domain experts and the software developers. However, that only works when this intense communication eventually makes at least one programmer a domain expert. So that strategy is more a way to reach this state rather than to circumvent it.

Whenever a software project works really well, there'll bee another interesting phenomenon: The developers acquire a deeper, more abstract knowledge about the work practices than the workers themselves.

extension3• 16 years ago

One that always seems to surprise non-programmers: there is no authoritative certification or regulation of programmers, the companies they work for, or the programs they produce. Being a software engineer requires no more credentials than playing the accordion on a street corner.

vog2• 16 years ago

> Bad programmers write code which lacks conceptual integrity, non-redundancy, hierarchy, and patterns, and so is very difficult to refactor. It’s easier to throw away bad code and start over than to change it.

Note that there's an interesting way in between. That is, refactoring code with the explicit goal of eventually replacing everything. There are several strategies to achieve that.

Those techniques proved to be less expensive and especially less risky than rewriting everything from scratch. They work because although the old code has an ugly design, it still has a property that is worth keeping: It is working and (mostly) debugged code. Undervaluing that property is the root cause of the many failed attempts to rewrite big software systems.

Some lesser-known truths about programming

Related Stories