Lets make the Objective-C Koans a reality.

http://www.puppysound.com/blog/2010/12/23/the-objective-c-koans.html

by thinksocrates • 16 years ago

11 2 16 years ago

(Apologies for the "Wall o' Text(tm)")

Unit testing is important where no prior culture of testing exists. However, where a culture of extensive testing already exists, introducing Unit Testing can be an inefficient way of moving resources from one kind of testing to another. Worse, since the proponents of it sing it's praises in an entirely uncritical fashion, adopting Unit Testing can lead to a net drop in the amount of testing, since if Unit Testing is so good why bother doing any other testing?

TDD's proponents will often claim that before they took up TDD their code was buggy, but now that they write code to test their code they no longer write buggy code... this seems to me to be a deeply deeply flawed argument.

So aside from a possible placebo effect, what are the real benefits of Unit Testing and TDD? Obviously someone that does no testing at all will receive a benefit if they start doing testing, even bad testing. As I write this of course it seems impossible to me that someone could write code and never test it... it just seems alien and bizarre. Surely someone would want to run their code to see if it worked???

Unfortunately, I have worked with people who don't have that inner spark, who don't even have the curiosity to run their code before claiming it is finished. The mind boggles, but it is true.

Okay, so there are people for whom a little more rigour, a little more testng would be a huge improvement. They will receive the most benefit from TDD and Unit Testing.

Then there are the people who just can't do design. TDD is a form of design. I wouldn't argue that it is the best form of design, but it is a design, so again, compared to someone not doing any design at all there is bound to be an improvement.

Strangely, the benefits I think for a developer with a mature process are twofold, and never mentioned by the proponents, which seem to me to be odd.

Firstly, even a mature developer can get code block or writer's block. TDD gives you a way out of it, because instead of writing 'proper' code, you just add another test. I believe many professional authors have a similar system, they might force themselves to write a certain number of words a day, and if they are really and truly blocked they'll just sit there and type "wibble" a thousand times on the paper - or until their mental processes rebel :D and the blockage is broken.

We could try to trick ourselves with some kind of 'programming-ish' make work to receive the same benefit (like twiddling the ant file), but our brains are good at detecting this sort of thing and slacking off. Whereas testing is both good and important, so it won't trigger those mental 'defences'.

The second benefit for a mature development process is that Unit Tests can be a good place to put your regression tests. The regression tests in a large corporate environment usually rely on a lot of knowledge that can get lost. Nobody might touch a big corporate project for 5+ years, and when you need to know all the gotchas, those people have moved on to different organizations or different projects in the same organization.

Joel Spolsky talks about how if you get a super obscure bug arising from some strange customer configuration, that having fixed it once, you should never release that same bug back out into the wild, and he makes a lot of sense. Unit Tests are ideal for protecting you from those sorts of blunders.

===

Any proper Pattern has consequences (check the GoF if you don't believe me :D )

There are two really big drawbacks to Unit Testing and TDD in general:

The thing the proponents will often claim as a benefit of Unit Testing, that it lets them make large refactorings on a large codebase at will, in order to fix their own bad design... really just makes me want to scream at them that they should have invested in a decent design up front, they should get better at naming variables. If you suck at design, get better at it. A programmer who sucks at design is like a carpenter that doesn't understand about the grain of the wood. I cannot imagine that such a person really cares about their craft.

On particularly 'Agile' projects, the Unit Tests are sometimes regarded as being place holders (or even complete replacements) for the requirements. Unfortunately what happens is that you get told to make a change. Making any change at all will break some test, often more than one. In a perfect world the trick is figuring out whether the test that got broken was because of the change in functionality, or whether it was because of an unforeseen breaking of some other requirement. Sadly, in practice what happens is that the developers get so used to 'fixing' the tests when they change the code that it becomes like a fish breathing water, they assume a test will break, so they stop thinking about why it has broken - and the test just become a tautological reflection of the code and cease to be guardians of correctness.

===

Disclaimer: I do way more testing than any Unit Test or TDD evangelist I've ever worked with. But I do a variety of testing. I believe any one form of testing most likely suffers from diminishing returns, so having a variety of different kinds of tests ensures maximum benefit. I also believe that no amount of automated testing can substitute for the eyeball test where a real human sits down and examines the output, whether that be via a gui (where applicable) or via some other means as appropriate. Automated testing generally only catches those errors that are predictable, whereas a good human tester can surprise you, and can spot errors that you didn't predict.

thinksocrates• 16 years ago

I appreciate the thoroughness of your comment. I also agree with some of what you said. For me, TDD is not so much about testing the code as it is about effectively writing the code. The only point that you brought up that I would like to make a counter argument to is the idea that you should get the design right on the first try. I don't agree that it is a good idea to design the system thoroughly up front. For me, using unit test to allow me to refactor throughout a project is very useful and allows me to become better at design everyday. It's an excellent way to prefect your skills as a code designer. It also leads to a code base that can be explained without any "well it used to..." sort of talk.

But again, I emphasis that I found everything else that you brought up to be perfectly valid.

Lets make the Objective-C Koans a reality.

Related Stories