Unit testing in Coders at Work

http://www.gigamonkeys.com/blog/2009/10/05/coders-unit-testing.html

by mqt • 17 years ago

109 85 17 years ago

jrockway3• 17 years ago

I sit somewhere in the middle. Unit testing GUI code is going to slow you down. Unit testing your core algorithms is essential. That is the stuff that must never break, and where breakages would not be immediately detectable. The unit tests ensure that you are always sure that that code works.

(Real world example; testing somewhat hairy parsing code: http://github.com/jrockway/moosex-runnable/blob/master/t/arg... I test every case I could come up with, so I could be sure that it works. Better to spend the time typing this file than to be hit with a weird misparse on the command-line, where you are probably not in the mood to fix the parser.)

On top of the unit tests, I do integration tests, to make sure sections of my app mostly work together. This is somewhere below "Request -> Response" and somewhere above the individual routines/classes. In the web app case, this is mostly "instantiate an object graph; call the methods that provide data to the rendering functions; make sure the results make sense". (I assume my template renderer and object database work; those have their own test suite, after all.) I also test stuff like typical sequences of actions that share some state (usually "the session"). I neglected that once and it caused me lots of problems.

But anyway, GUI testing == waste of time. As long as the buttons you click generate the right events, the GUI works. This is the sort of app Zawinski was working on, and this colors his thoughts on testing accordingly. Your mileage may vary.

brown9-2• 17 years ago

Why exactly is unit testing of GUI components a waste of time?

jrockway3• 17 years ago

Because it takes a lot of time and effort but does not provide much benefit.

chrisconley• 17 years ago

Maybe a better way to put it is that testing GUI components is not a complete waste of time, but it gives you a lot less bang for the buck.

silentbicycle3• 17 years ago

The part about TDD in the interview with Peter Norvig really jumped out at me, too. A passing test doesn't always mean anything important has changed - like most other programming tools, there are circumstances where unit tests just create the illusion of progress. I also remember Norvig saying* that sometimes the scope of pass/fail tests may be too narrow, because sometimes having 18 of the first 20 results be reasonable is good enough, but typical testing packages aren't really structured to accommodate probabilistic thinking. Certainly more relevant for some problems than others, but interesting nonetheless.

I've seen major benefits from automated testing, and I'm convinced they're a net win (especially for maintenance), but I also think that being dogmatic about any particular testing methodology (TDD, etc.) is going to be counterproductive sometimes.

* I don't remember if it was from the same book, PAIP, AIMA, or something else - I've been bouncing around in several AI and Prolog-related texts lately.

wglb• 17 years ago

The Norvig points were particularly thought-provoking. How do you tell if google is returning the correct result.

The other is that starting a unit test without knowing what you are aiming for (e.g., the puzzle solver) you may not get there, and TDD won't help you.

keyist• 17 years ago

If you enjoyed seeing the contrast in approaches between Norvig and Jeffries, you may also find the following interesting -- calculating bowling scores in

1. OCaml, no tests: http://alaska-kamtchatka.blogspot.com/2009/07/disfunctional-...

2. Clojure with unit tests: http://blog.objectmentor.com/articles/2009/07/19/uncle-bob-j...

</amazon>

tptacek3• 17 years ago

Unclear on how anybody can cite the author of the latter with any kind of reverence, but I see that happen all the time.

mhartl2• 17 years ago

This seems unfair. (a) OCaml != Clojure; maybe the former is better for this problem. (b) Uncle Bob writes I’m trying to learn Clojure, i.e., he admits he's brand new to the language and doesn't (yet) know what he's doing. So, maybe cut the guy some slack?

mahmud4• 17 years ago

Enlighten us folks who aren't methodology scenesters. What is wrong this Uncle Bob guy?

dagw• 17 years ago

One thing I've never really seen discussed is proving unit test coverage. Taking the clojure example above and its unit tests, nowhere does it make any attempt to show that the few test cases chosen provide a complete coverage of all possible bowling scores. Without that what have you really shown? Sure I can look at it and my gut feeling is, yea that should be enough, but I got that from looking at the OCaml code as well

In the blog post Brad Fitzpatrick is quoted as saying "Write a test. Prove it to me." Now maybe it's my maths background, but when someone say "prove it" a expect proof, and the bowling unit test example proves nothing more than that it works in a few common cases.

Yes I realize that proving anything non trivial in programming is Very Hard, but without any sort of reasonable attempt and even talking about this problem TDD seems more or less ad hoc to me.

eru5• 17 years ago

You should have a look at QuickCheck. It's a Haskell library where you (more or less) specify laws that your program should satisfy, and it generates test cases on its own. I found it very useful.

extension• 17 years ago

Proving test coverage is just as (in)feasible as proving correctness. This is the elephant in the room for testing: it relies on programmer intuition to enumerate cases. It's easy to neglect a case both in the implementation and in the test. This is why tests are better at finding regressions than new bugs.

icefox• 17 years ago

I can't "prove" my code, but I can prove that my test executes every function, every line and takes every branch and every return. I run my test using valgrind (callgrind with a few options), then I wrote a valgrind parser and combined that with a c++ parser and output the results. Some fun tidbits:

Every time I discovered that a bit of code can never be executed.

The amount of functions that I never even call, even when I am using my test generator (built on top of the same c++ parser)

I enjoy coding. If I spent 1 hour coding a test that finds nearly all of the bugs in a class that means later I wont spent a week deciphering reports and fixing bugs that are annoying users. I get to hack more and my users get more stable code.

If you want to try out my code, http://arora-browser.org/

The c++ parser I used http://github.com/icefox/rpp/tree

My evil little test generator http://benjamin-meyer.blogspot.com/2007/11/auto-test-stub-ge...

My valgrind tools http://benjamin-meyer.blogspot.com/2007/12/valgrind-callgrin...

Gupie• 17 years ago

I know which I would like to maintain.

plinkplonk8• 17 years ago

fwiw, I wrote the original blog post (that Norvig was speaking about in his interview) which compared the Sudoku efforts of Norvig and Jeffries) - http://ravimohan.blogspot.com/2007/04/learning-from-sudoku-s....

projectileboy• 17 years ago

That was a fabulous analysis. For my part, I can't quite figure out why people seem to get emotional about TDD one way or the other. If it helps you, great. If not, ditch it. Do people work in environments where they'd like to avoid TDD but are forced to work that way? Or vice versa?

stcredzero2• 17 years ago

Do people work in environments where they'd like to avoid TDD but are forced to work that way? Or vice versa?

Yes. Unfortunately, it's one of those things that only works well if you have total buy-in from everyone on the team. Maybe we should avoid methodology that requires too much of that. Reminds me of how communal living only works if everyone pitches in. How about a methodology that works entirely off of inherent human laziness and self interest? XP works off of some laziness. But it's more like Air Cavalry in Vietnam. Land the soldiers behind enemy lines, and they have to do something productive just to save their own butts. Likewise, write some failing tests, and you have to code something to get the tests to turn green. (pass) But the problem is getting the soldiers on the choppers to begin with, or getting developers to write the tests. In the Army, there's the threat of the stockade and the firing squad. In XP, it's just getting fired.

Another problem that startups are supposed to be able to avoid.

nostrademons3• 17 years ago

Can't the teams just decide this by consensus?

There're a bunch of these practices that require total buy-in from a team. Things like "Are we going to use bugtracking software to coordinate who's doing what?" "Are we going to have daily stand-ups?" "What'll the conventions be for X?" etc.

On the projects I've been on lately, the answer has usually been "Fine by me. Let's give it a try and see if it works out." And if it's not working out, we don't do it anymore. But oftentimes they really do help, and we end up introducing the same practices to our next project.

johnrob• 17 years ago

I generally take a middle road - I write code that is unit-testable, but I rarely take the time to write exhaustive tests. When bugs arise, I start writing test cases in various components until I find them. Thus, the debugging effort is what grows the test coverage. Of course in a vacuum it's better to have the cases earlier rather than later, but I like this approach as a speed/quality compromise. The key to making this work is designing in a test friendly way - that is the true art.

earl2• 17 years ago

I'm personally of the opinion that this comprises a large portion (>60%) of the value of unit tests. Unit testable code tends to not have many of the side effects / global variables / nasty state that make nasty bugs, so this alone tends to drive down the number and severity of bugs. The testing itself is icing on the cake.

wenbert• 17 years ago

I strongly agree with this. The sad reality is that some of us have deadlines to meet. I think this is an option that makes sense. Writing testable code to get the job done and meet the deadline and the unit tests come in later when needed. EG: new features, bugs, etc.

Not a good practice and ideal but it is practical. IMHO

eru5• 17 years ago

It's not too bad. At least you avoid recessing bugs.

wastedbrains• 17 years ago

I do that a lot to, I try to call it Test Focused Development opposed to TDD

floodfx• 17 years ago

Unit testing is the only way to scale lines of code without scaling your team.

Aleran• 17 years ago

In all fairness, how can anyone compare Peter Norvig and Ron Jeffries on the same level?

One is an AI genius and the other is a XP coach. They are on very different levels intellectually.

mquander• 17 years ago

Solving Sudoku isn't exactly rocket science. I can understand if you don't have the cleanest, most elegant solution in the world, but if you can't even make a tiny bit of progress toward writing a Sudoku solver in a few hours, then I don't think you have any business holding a programming job, much less telling other programmers how to do their jobs.

I mean, those Ron Jeffries blog posts read like someone who has never solved a non-trivial programming problem in his life. He literally makes no headway on any difficult part of the problem, and he spends what appears to be the better part of several hours working hard to get nowhere on code that does very little. If someone is listening to him about how to approach programming projects, I've got a bridge to sell that man.

litewulf• 17 years ago

In fact, solving Sudoku puzzles was a programming assignment given in my first year of undergraduate study. We were given input/output specifications, and told to write in C++. That was it.

Seeing as how most of the class passes, I assume almost any joker can write a half-decent sudoku solver if sufficiently motivated.

abecedarius• 17 years ago

It's not just that Norvig is smart; he's specifically skilled at writing really good code, apparently because that's something he cared about and worked on. At JPL I worked with a bunch of smart researchers with PhDs but some wrote better code than others, and it didn't mean they were on a different level intellectually.

Norvig's book _Paradigms of AI Programming_ has 900+ pages presenting code as instructive as that Sudoku solver; I've never seen a better collection. http://norvig.com/paip.html

earl2• 17 years ago

Dude, a sudoku solver is really directly solved via brute force search. If a coder can't come up with the brute force algorithm for a 9x9 sudoku board, there's something wrong with that person.

eru5• 17 years ago

Brute force? You can write a solver in 15 simple lines of Zimpl code (pun intended), that uses state-of-the-art constraint propagation and all that.

http://www.zib.de/Publications/Reports/ZR-05-51.pdf

mrshoe• 17 years ago

The most important lesson to learn here is that all code, including tests, is a means to an end. A lot of people geek out about new programming languages, object oriented design, automated testing, TDD, XP, and various other methodologies du jour. If those things help you achieve your goal, that's great; but it's important to remember that they're (usually) not your goal in and of themselves.

I don't think jwz would say that unit testing is a bad idea. What he was trying to say when he dismissed them was that they were focused on making a browser, not on making a nice piece of code.

Similarly, Norvig was focused on solving sudokus, not on exploring methodologies that might be used to write the code which solves sudokus.

plinkplonk8• 17 years ago

"TDD, XP, and various other methodologies du jour. If those things help you achieve your goal, that's great; but it's important to remember that they're (usually) not your goal in and of themselves."

When you are an agile "coach", Scrum Trainer etc, TDD, XP , Scrum etc are your goals!

mhartl2• 17 years ago

Nothing can excuse testing a non-solution, so I won't defend it. But correct code, code that does solve your problem, isn't always pretty: sometimes it's straw instead of gold. And testing is orthogonal: you can write untested gold, untested straw, tested straw, or tested gold. Sometimes you're lucky and untested gold bursts fully-formed like Athena from the head of Zeus. Otherwise, be glad for testing, which lets you turn untested straw into tested straw, and thence, via a refactoring Rumpelstiltskin, to tested gold.

Tichy• 17 years ago

"One thing I noticed, reading through Jeffries’s blog posts, was that he got fixated on the problem of how to represent a Sudoku board."

Reminds me of pg's writings on exploratory programming (I think that is what he called it).

For me, maybe I am a bad programmer, but I always change my mind on how to represent things once I sit down to code. I can read specs or think about how to do stuff over and over, it all only happens when I finally start to code. So it sounds as if too much test first would make me stuck. Plus, as the article points out, it would be boring to always just work on unit tests and never actually produce code that does stuff.

Not that I dislike unit tests, but I tend to write many of them after coding the main stuff (in the testing phase).

bad_user2• 17 years ago

I simply loved this article and the comparison made between the two Sudoku solvers.

Peter Norvig is one of my favorite programmers ... his solutions are always simple and elegant. Have a look at his spell-checker here (a solution I used in production): http://norvig.com/spell-correct.html

He just kicks ass, and it reinforces that people (at least in software development) are more important than processes.

I don't do TDD because our consultancy gigs only last for 2 months tops, during which we have to do what other firms are doing in 1 or 2 years. We are also most of the time in uncharted waters, and doing proper unit-testing of such modules requires even more thought than the actual functionality. And after two months tops, the project is over, and we won't get any money for extra work. We are doing unit-testing for really critical sections though.

But I do wish I had the time and the skills to do TDD.

whichdokta• 17 years ago

When testing helps, test. When testing hinders, stop testing.

This business of "must" and "always" anything ?

Not so much.

Unless your business is a lucrative hourly rate telling those who believe in "must" and "always" what they should be doing instead of thinking for themselves.

DanielBMarkham• 17 years ago

Great article. Lots of detail and the author doesn't get into editorializing too much.

I'm in the "want to believe" category with TDD, solely for the reason that I think there are a lot of smart people advocating it. Having said that, I haven't seen a lot of teams that use it and continue using it for a competitive advantage -- and that sets off alarm bells. The hype may very well be ahead of reality on this one. I honestly don't know.

10ren• 17 years ago

One day I hope Joel eventually realizes this. Programmers who say they don’t have time to write tests are living in the stone age.

This kind of attitude really pisses me off. It's so confident in its condescension and name-calling, I have a little wonder that they might be right. In fact, it's just dogmatic abuse, unencumbered by an objective factual appraisal of the issue at hand. Probably, I should just remember that dogmatism tends to be inversely related to wisdom.

For one thing, the purpose of the code makes a tremendous difference: e.g. unit-testing is great for maintainability, but terrible for evolving an API (as in prototyping or exploratory code).

And thank goodness for Knuth.

ZeroGravitas4• 17 years ago

I was amused by his comment on Knuth:

"So Knuth too disagrees with the notion that unit testing always makes you go faster. Maybe he too is living in the stone age."

This follows him describing writing a program, in pencil, in 1977.

briancooley• 17 years ago

I think the list of programmers throughout history who could reliably write something as complex as TeX using nothing but paper and pencil is incredibly short.

nickelplate• 17 years ago

What I find most irritating is that the TDD or other methodology zealots usually cannot point to a success of exactly the sort people like jwz or Joel have been influential in shipping. There also seems to be a tendency among these people to confuse rejection of TDD with rejection of unit testing.

jerf• 17 years ago

My position on unit testing is, "Have you tried it?", followed by, "No, seriously, have you actually tried to use it for a period of time, not just played with it for an hour?", which I will then follow by... wishing the programmer well regardless of how they now feel about it, because now they've at least got some experience.

Unit testing is a big deal. Everybody should try it. Ideally, you should try it with the next project you start from scratch (or subproject as the case may be) because trying to retrofit an existing code base does not give accurate impressions. I'd also point out that some things are easier to test than others, but that once you have a bit of experience sometimes even the hard ones turn out to be feasible. (For example, you shouldn't go straight to UI testing, but there are many other good testing starting points, like parsing or network communication.) You really should stick with it at least long enough for it to detect at least one bug that is a total surprise to you, because that will happen, it's only a matter of time. But once you've tried it, I respect your opinion after that.

I've also tried TDD, which I just could not get into, whereas unit tests I love.

silentbicycle3• 17 years ago

It's easier to learn to use the testing tools themselves on an artificial project, but I find that trying to build testing into an existing project teaches you different (and perhaps more useful) things. It really brings in to focus how to write (or rewrite) code to be testable, and you learn much faster when it's actually a huge win vs. when you would just be going through the motions. If a technique only works on (say) blog-post-sized Ruby projects, then that should be a red flag. Testing is a means to an end.

plinkplonk8• 17 years ago

" "One day I hope Joel eventually realizes this. Programmers who say they don’t have time to write tests are living in the stone age."

This kind of attitude really pisses me off. "

Then you'll really love "Uncle" Bob's latest blog entry (http://blog.objectmentor.com/articles/2009/10/06/echoes-from...) where he re iterates his inane "Stone Age Programmer" name calling. The comments on the blog post are more interesting (well ok, funnier) than the content of the post.

tetha• 17 years ago

What I hate most about this article is that they stress the "always". It is impossible that unittesting is faster and better in every possible software for every possible hardware in every possible situation in this universe and time, just plain impossible.

Certainly, it will make you faster in a lot of cases (and a lot of regular buiseness-programming is among these cases) but there are certainly cases where it is wrong. Think of a code emitter in a compiler, or handcrafted assembler for some micro controller, or especially a problem where you just know the solution, and it is simple. In these cases, it just is faster to just hack down the solution.

ZeroGravitas4• 17 years ago

So the general conclusion, from various programming legends, seems to be:

• (semi-)automated testing for any complex code: essential

• unit tests for any long lived code: great

• TDD (writing the tests before the code): good in theory but alien to most folks

• tests speed/slow you up/down: open question (though it would seem logical that there will be certain projects that benefit more than others, e.g. the longer lived the code the longer you can earn "interest" on your initial "investment")

Yet there still seems to be a general reaction here of "I'm too smart and/or busy to write tests" and the corollary "Anyone who writes tests, or advocates testing, isn't a good programmer".

Seems a strange disconnect.

plinkplonk8• 17 years ago

"(semi-)automated testing for any complex code: essential"

Where did you see this idea in Coders At Work? Most of them say testing is important . Most of them don't say "semi automated" testing is "essential". So where is your perceived "consensus" of programming legends coming from?

or this

"TDD (writing the tests before the code): good in theory but alien to most folks"

for that matter? Sibel mentions 5 developers in his blog post - Zawinski, Knuth, Norvig, Bloch, Armstrong. I didn't see anyone saying "good in theory but alien to most folks". Only Armstrong says he regularly does (soemthing like) "test driven", but if you read his interview he also says he spends a lot of time doodling and getting his abstractions right before coding, hardly the "YAGNI" style of coding and theory of "emergent design" propounded by most agile consultants.

Who in CaW do you think concludes either of the two statements you put forward as "general conclusions from programming legends".

Or are you saying Robert Martin and co are programming legends? Bit hard to justify I'd think ;-)

You say,

"There still seems to be a general reaction here" .. "I'm too smart and/or busy to write tests" and the corollary "Anyone who writes tests, or advocates testing, isn't a good programmer"."

The former " I am too busy to write (TDD style) tests" is a justifiable argument in certain circumstances (Zawinski says as much) and the latter "Anyone who advocates testing is a poor programmer" is something I don't see anyone here saying. Who said this? Links please? And if someone did, why do you think it is a "general reaction"?

fwiw I see the evolving consensus here ) as "writing unit tests is valuable in certain circumstances (and not so in others) as long as you aren't fanatical about them" and "what is important is not the methodology but the end result".

You seem to be putting forward your beliefs about the value/validity of TDD/testing etc as the "general conclusion, from various programming legends". I am curious as to how you came up with this apparent consensus.

Don't get me wrong, you certainly have a right to your conclusions. I am just saying the CaW interviews and the posts here don't seem to converge to a conclusion along the lines of you say they did.

ZeroGravitas4• 17 years ago

I got all this directly from the linked article:

Zawkinski: created automated tests for a complicated corpus of quasi-standard email headers that he triggered manually. When he rewrote the mailer in Java he unit-tested because it was easier in a Java OO environment. He says there's specific areas where unit-testing would have sped them up, but that they were under so much time-pressure, with short-term ROI horizons that they didn't bother, they just went with the gamble of "getting it right first time". If they had more time then unit-testing was "certainly the way to go". He specifically states that they traded quality for a ship date.

Fitzpatrick: Does "a lot of testing". Tests anything half-way "clever" because others will break it. Forces people who work for him to write tests. "At a certain point, people realize, “Holy crap, it does pay off,” especially maintenance costs later."

Joshua Bloch: automated testing of transactional shared-memory with a "basher". When it failed at high concurrency he had to use unit tests to locate the bug, which wasn't in his code but the underlying Solaris assembly code. This was early 90s and he implies that finding a bug in such code today would probably result in the engineer being shouted at for inadequate testing, though perhaps not in concurrency areas as testing those is still an "art". Argues that writing an API before the code, which he does, isn't technically TDD though I'd say it's in the same ballpark.

Knuth: is "a fan of after-the-fact torture tests".

The blog author interprets the other comments as anti-TDD, though they seem unrelated to me.

Joe Armstrong: Actually does TDD.

Norvig: "I think test-driven design is great. I do that a lot more than I used to do."

I don't understand how you could have come up with a different reading. The originally post seems to be half-heartedly questioning TDD (without much support from the quoted sections), but that is on the far end of a continuum that has automated tests, then unit tests, on the way towards it. Which I thought I made clear with my list where the near end of that spectrum (automating tests for complicated areas) is better supported than the far end (TDD according to the book). Many responses I see here seem to reject testing entirely.

(edit: just to be clear when I say "automated" I mean they wrote a program to test their program. It doesn't necessarily mean that they run it on every change, or check-in or whatever. I just mean it runs and says: pass or fail without a human being having to examine the output, though that output may be helpful when it does fail.)

ErrantX2• 17 years ago

I agree with Spolsky on this one; and I cant help feeling TDD is like agile. One of those things cool companies and corporates use to sound "cutting edge".

We used TDD for a few projects (and agile too) and found it does slow you down no end.

And it also ultimately doesn't catch many useful bugs and problems. We still had to go through the usual end-point testing cycles.

We write unit tests for any of our API's and also for some of our core code; but only after original versions are in place. They are supposed to catch any mistakes or errors in futures we enhance and evolve the programming so we don't break backwards compatibility.

I think it really does come down to what works best for your teams though. Im sure plenty of people find TDD a great addition to their arsenal.

But at the end of the day it's just a buzz word like everything else - and we've found that by avoiding those kinds of things we produce good, solid, working code at a fast (not necessarily faster pace) - but with less headaches :)

gaius• 17 years ago

The thing with unit testing is, a lot of the time it's used to try and make a weakly-typed language with uncontrollable side-effects behave like a strongly typed language in which pure functions can be written.

A team lead who doesn't understand the above sentiment is at best only going through the motions with unit testing.

ErrantX2• 17 years ago

Im not sure I agree.

Unit testing is about code being to a specific standard (i.e. that API works like X or like Y).

Surely your referring to Fuzz testing?

gcv• 17 years ago

As far as testing goes, I have a simple approach.

If I just wrote some code and had to poke at it in a REPL with some different inputs to check if it works, then I should take whatever I did and turn it into a test. With a good testing framework, like the one built into Clojure (formerly clojure.contrib.test-is and now clojure.test), it's really easy. I just copy from my REPL and paste it into the appropriate test file, along with the expected results.

If I find a bug, I fix it, write a test for it, and also copy whatever I did to check my work from the REPL into a test file.

That's it. I routinely change code around with all the confidence TDD advocates claim. I don't test trivialities. I don't test if my database connection was correctly established --- but I will test database connection loss error handling if I put any non-trivial logic in there.

I'll often write a little snippet of sample code for an API I'm working on, but I never write a full formal test in advance. It makes letting the actual API evolve as it grows too cumbersome.

eru5• 17 years ago

Sensible.

Though writing a test case for an evolving API might be of benefit from time to time, because it forces you to use the API (and thus see if it is sensible).

maryrosecook2• 17 years ago

You know what has improved the quality of my code? Having every error on my live servers emailed to me. I feel compelled to fix it because I know that an actual user has had a problem.

I tried to do unit testing for a project. I'd write a method, then write some tests for it. Some observations:

* I thought that writing the tests would give me refactoring ideas for the actual methods. It did not.

* Running my test suite prior to each deploy saved me from shipping a bug once. Once.

* Writing tests is fucking boring.

This was all code that I knew the purpose of in advance. I have no idea how one writes tests when doing exploratory coding.

geebee• 17 years ago

"I have no idea how one writes tests when doing exploratory coding."

You're probably already writing tests, especially when you're doing exploratory coding. How much code do you really write before you check to see if something works? You're almost certainly doing something. Maybe it's just a small main program where you validate that some methods are giving you the output you expected, or maybe it's a simple web form that you populate with mock data to verify it's persisting to a database. Unless you wait until the entire functionality is ready before a trial run (and if you're doing exploratory coding, there's no such thing as "ready" anyway), you are writing something to test during iterations.

Instead of throwing these tests away, keep them somewhere and run them periodically. If they break, figure out why - and either fix the code, or change the test. When you're done, you'll have a bunch of unit tests.

(just a quick note - the process I described above works extremely well for some situations, but poorly for others. Some frameworks make it a real hassle to write unit tests. The process I described also is not TDD, it's more of a write a little, test a little approach, which feels more natural to me. I also don't really like to use TDD, especially for exploratory coding).

maryrosecook2• 17 years ago

You're right: I write a little, do a test run of the code to see what comes back, fix it if it doesn't work, write a bit more and so forth. And I see how saving those little tests could be useful.

toadpipe2• 17 years ago

So now that I've voted up every plinkplonk comment, my comment is that most people obviously haven't read the book. The post is pretty good, but the book is better. You should read it.

If you do read the book, you will see that the discussion begins with Norvig saying that he thinks one of the most important things is being able to keep everything in your head at once. Extra tools come in when the problem gets too big to do that, but here's the key: he saw right from the very beginning that the Sudoku problem could be solved with two tools from the AI toolbox. In other words, he saw the entire solution immediately, and it was never too large to fit easily in his head.

Seibel's analysis fine as far as it goes, but he misses the really important thing here, which is that this is not just an example of someone recognizing a problem that they already knew how to solve. It's a case of someone with the mental tools that allow them to dramatically reduce the (apparent) complexity and size of a very large class of problems that happens to include Sudoku. Seibel takes a bottom up look at the data structures Norvig used, but this can be misleading because they weren't designed bottom up, they were designed all at once. You simply cannot do that unless you have the necessary training in abstract thinking (read: mathematics/formal logic/language development/ai techniques/etc).

No amount of code centric techniques or tools will ever make up for not having these tools. There will always be (relatively trivial) problems that you will never be able solve without them, because you will not be able to fit everything in your head and your ability to reason about the problem will be crippled by that. Debates about TDD are not even wrong, because they are at the wrong level of abstraction. Spending any significant amount of time discussing it is premature optimization.

Unit testing in Coders at Work

Related Stories