Reasons to Avoid Test Driven Development

http://www.softwareandi.com/2011/12/10-reasons-to-avoid-test-driven.html

by LiveTheDream • 14 years ago

39 40 14 years ago

CJefferson2• 14 years ago

This article is just irritating sarcasm.

There are perfectly good reasons to avoid test-driven development. Like being part of an existing large project, where there are already too many interconnected components. Or working in complex AI algorithms, where separate testing of components is extremely painful, and lots of whole system testing is necessary anyway.

benjaminwootton2• 14 years ago

Could you expand on this?

Why are too many interconnected components or a complex AI algorithm a valid reason not to test or use test driven development?

My view is that in a complex system, only by having confidence in the units can you have confidence in the integrated whole.

You might be able to build a house that stands and looks nice, but if the timbers, the bricks, the foundations etc are all flawed in their own small ways, sooner or later those problems are going to combine and you will end up squashed.

I can't think of a single example where automated testing couldn't potentially improve what you deliver and the process of working on it. Even the hackiest throwaway script could likely be produced faster and easier with a little automated driver.

plinkplonk• 14 years ago

"I can't think of a single example where automated testing couldn't potentially improve what you deliver and the process of working on it.

Not engaging with the above argument, just trying to clarify an ambiguity. Just saying (and trying hard to keep tone neutral).

automated testing != test driven development.

The former has been in use for decades. The latter is a relatively new methodology/practice which emphasizes writing tests first and using tests to drive development.

CJefferson2• 14 years ago

Oh, we certainly do massive automated testing. Our release testing script takes currently an 8-core machine about 4 days, and our nightly test runs on about 6 configurations, taking about 2 hours.

However, often the units have properties which are very difficult to specify in isolation, and from experience almost all bugs come from many algorithms reading and writing the same data structure (I'm not thinking of threading issues here, although they are another issue).

Consider something like a sudoku solver (the things I work on are much more complicated than that). Usually writing the function which checks the condition 'a row/column/box are all different' is very simple. The errors come when these functions interact, each effecting each other as they are called.

Certainly, this kind of situation is very unusual. Most people write code which lends itself more naturally to test-driven development, and in those situations I approve of it.

lucisferre2• 14 years ago

Agreed this was completely pointless. Bottom line is not every line of code needs testing, and TDD alone does not have an impact on code quality. There is nothing I hate more than bad code covered by even worse test code.

Here is a talk for anyone who is wondering why TDD doesn't make sense as a default mode: http://www.youtube.com/watch?v=LeVvj4HENOQ

Bottom line is that writing TDD style unit tests won't make crap code better any more than removing them makes good code worse. TDD is a tool, nothing more, nothing less.

rhizome• 14 years ago

Hmm...a 90min Youtube will have to wait until I'm done writing tests.

tzaman• 14 years ago

Not entirely true: If you write tests, chances are you're educating yourself on how to write them. As a consequence, code gets improved - because it's almost impossible to write any kind of tests if the code is crappy. But then again, maybe that's just me :)

__david__2• 14 years ago

Also, code that controls hardware is very hard to test.

Consider writing tests for the the Linux kernel's device drivers. It would be very difficult because most of the problems have to do with hardware being buggy/underspecified, not with the software being wrong in a testable way.

In order to test a UART driver you have to to simulate the way the UART worked, which means that you wouldn't catch bugs if your understanding of the hardware was wrong. Most of the time spent would be recreating the internal logic of the UART and it wouldn't actually benefit you unless you got it right.

The other option to test the UART is to have a rig set up that where you can run the code on the device and have the serial port looped back or go into another computer that checks the result. But that (by definition) means special hardware is required to run that test, which makes it way less useful as an automated test.

Neither of those situations is amenable to TDD.

mpweiher4• 14 years ago

There's actually a fairly simple way to deal with external systems that to me would seem to apply to hardware as well:

First, write code that "tests" the actual hardware, that means it drives the hardware and checks its results. Verify using whatever means you have available that it drives the hardware correctly.

Second, create a variant of this first test (class) that runs against your simulator (checking for the same results). I used a trivial subclass that changed nothing except replacing the device interface with the simulator.

Third, write your simulator against these tests.

Fourth, use your simulator to test the remainder of your system without requiring the hardware/external system.

Extend when you make use of new features, and occasionally test against the real system/hardware...

codewright• 14 years ago

The tl;dr (it's a software product I think?) for this article is a dead-pan recitation and is hilarious.

lancefisher• 14 years ago

I thought it was going to be sarcasm, but clicked through anyway hoping it wasn't.

Sarcasm is annoying and often hurtful. I've tried to eliminate it from my everyday speech. Maybe that's a reason this article irks me, and I like TDD.

SoftwareMaven• 14 years ago

I was disappointed by this article. There is no such thing as a "perfect" methodology, so I tend not to trust people who tell me their methodology is perfect. I have a lot more respect for people who recognize the tradeoffs that any methodology forces and then finds the right places to use them and the right places not to use them.

And this is not to say anything at all about TDD. There are a lot of good uses for it. It just isn't the One True Paradigm of software development. But then again, neither were dynamic languages, object oriented programming, extreme programming, nor any of the other paradigms that were going to cure the problems of software development that went on to add value when creating well thought out plans for every unique project.

netmute• 14 years ago

The article lists 10 reasons which can basically be boiled down to:

1) Unrealistic assumptions ('perfect' developers, 'perfect' architecture etc.)

2) You simply don't care about quality.

jonny_eh• 14 years ago

I think it was a dry satirical humour piece.

Aco-• 14 years ago

somehow I doubt that

zorked• 14 years ago

I find it telling that proponents of TDD so often resort to name-calling and sarcasm to promote their views instead of, like, just writing great software.

kevinw• 14 years ago

I'm a bit confused why this article was written in a satirical (sarcastic?) style. It raises a lot of good points, but it would have been much more straightforward if it hadn't been written sarcastically. All of the points in the article could have been written from the position that the author ACTUALLY advocates, which would have made it more readable without sacrificing any content.

jakejake• 14 years ago

It seems to me the author is such a proponent of TDD that the idea of not using it is laughable.

The humor is so dry that it almost seems like a serious article. I got a few laughs though.

benjaminwootton2• 14 years ago

I was about to ask if it was satire. Guess it was wasted on me!

ludflu• 14 years ago

I'm a strong proponent of TDD. In my experience its more productive, and results in better software. But....

I've been in this position before: We have a large existing codebase with no unit tests to speak of, and the business is not willing to fund efforts to refactor or add unit tests. It is very hard to add unit tests after the fact. Not impossible, but its alot of work, and business folks really don't care.

rymith• 14 years ago

I've added unit tests to an existing legacy product, and it would have been simpler and faster to just re-write the application with TDD. The problem is that a project not written with TDD in mind often is very very difficult to decouple, and you end up either having essential sections without sufficient coverage, or you have to come up with come pretty abstract and fragile methods to test with. It's a mess.

atte• 14 years ago

In my experience, the way to approach this is to start writing tests for any bug fixes or new features added. Over time, coverage increases and the product becomes much more solid.

manojlds• 14 years ago

TDD is test driven development. You are talking as though TDD is same as adding unit tests.

mpweiher4• 14 years ago

I think he is describing the (common) situation where you want to do TDD for new code, but are dealing with an already existing code-base that doesn't have test coverage and just isn't amenable to testing the way it currently is.

So you can't really do TDD for your new code without refactoring the existing code-base. But refactoring the existing code-base without good test coverage is nasty.

nullc• 14 years ago

Some useful arguments but the sarcasm isn't needed…

And it belittles some of the complicated arguments such as: If you develop to the test, the test will always pass— but that doesn't mean that your software is any good. Loosely coupled development and testing can result in the tests being more powerful proxies for overall quality.

colomon• 14 years ago

The article doesn't seem to recognize there is a distinction between TDD, having unit tests, and having automated tests. Personally, I've used all three approaches, and think each has its place. As nearly as I can tell, anyone who says one approach fits every programming project must not have a very broad experience with different sorts of projects.

tyre• 14 years ago

These are reasons for testing, not TDD. There is no discernable difference between 2 shipped products with identical test coverage where one team used TDD and the other just wrote regression test.

The worst part about TDD is the people trying to shove it down your throat.

geebee• 14 years ago

I think you've identified one of the weakest parts of the essay. This is about ignoring testing, not TDD.

One way I've convinced people to write unit tests is to remind them that they are already testing. People often write a little main to check the output of a program. This is a test. People often click on a link to validate that some text is present. This is a test. Keep them! Instead of throwing away your stand alone program, put it in the unit test folder. Instead of throwing away that point and click, automate it and put it into the integration tests folder. This alone will get you close to 100% coverage. Now, when you refactor your code and it no longer works the way you verified it would a while ago, you'll get a little red bar or a "FAIL" message. Usually this is because you wanted the behavior to change, so update the test. Occasionally, this is because you introduced unwanted side effects in your code, which is exactly why you want these tests.

TDD is substantially different from what I just described. If the author wants to defend it, that's fine, but let's make sure we understand it's not the only way to get high levels of code coverage.

programminggeek2• 14 years ago

TDD is awesome, if you have a codebase or a tech stack that makes it nice. TDD will quickly expose bad design through an unpleasant testing process. For example, slow tests, untestable code, or tests that you "always have to rewrite/maintain" are problems that only show up when you try and TDD your code that isn't designed to be testable.

TDD doesn't work well if you don't design your code to be testable because you very quickly become frustrated by the problems TDD uncovers.

1. Slow Tests - If your tests are taking a long time, it's probably because your code isn't written to be tested easily.

For example, if your code is hitting the database a lot and it starts to take minutes or hours to run your test suite, your code has a problem. For example, if your entities/models are coupled to an ORM like ActiveRecord, testing those is going to suck because they probably sholdn't be coupled to an ORM. Many people try to fix this by writing frameworks that make testing their code easier instead of writing better code that isn't tied to an ORM.

Tests should be fast. Most of the time they don't need to hit the DB. Slow tests make TDD suck. Avoid slow tests.

2. Untestable Code - I've run into this a lot and it sucks, especially in languages that lend themselves easily to untestable code (In my experience, that's PHP).

Until you try to test your code, you don't realize how untestable your code is. A lot of things that seem like reasonable design decisions are so untestable, that you end up doing incredibly lazy integration tests because to fix things you have to rewrite whole portions of the system.

I've been there and it sucks. The choice seems to lie between rewriting the whole system on the same platform to be testable one chunk at a time, or to rewrite the system in something that lends itself better to testing.

In either case, the answer is to take the time to learn how to write testable code. To start learning techniques like Dependency Injection. To practice writing portions of your system using TDD to see what works and what doesn't.

3. Maintaining Tests Sucks - Actually, if you don't want to maintain a test suite, don't write one.

At its core, your tests are a specification of how your system should work at a given time. When they start breaking it means your system has likely changed, so either it should be fixed to match the specification, or the specification should be changed to match the appropriate system behavior.

At a fundamental level, tests aren't something you write once, they are something you constantly rewrite to keep them in line with what your system should be doing. Thus, when they are broken, you know that either your specs need to change or something in the code isn't working right.

Test maintenance is a price you pay for wanting higher quality software. It's the same kind of cost you pay in designing a building with blueprints. If people build a building without following the blueprints, it's not going to be the building that was supposed to be built. Tests are a living blueprint of your system. If you aren't willing to treat it that way, TDD isn't going to help you.

TDD isn't for everyone because not everyone is willing to make those kind of investments in code quality and maintainability. Also, TDD doesn't make your code better if you aren't willing to let it make your code better.

__david__2• 14 years ago

> TDD isn't for everyone because not everyone is willing to make those kind of investments in code quality and maintainability.

One thing I've noticed is that code written to be testable tends to suffer from over-abstraction. Sadly, writing nice, compact, easy to read code is somewhat at odds with TDD in my experience.

I personally don't like that I have to make sacrifices to code simplicity just to make it testable. But perhaps I just suck at TDD. :-)

programminggeek2• 14 years ago

I totally understand what you mean.

I've gone to great lengths to determine what kind of architecture is needed to truly support TDD and what I've come to realize is that it's not so much about over-abstraction that is the problem, it's that testable code is pluggable, which is mostly at odds with how systems are designed.

For example, standard MVC apps tend to tie models to an ORM in some way shape or form. Think Rails and ActiveRecord and you'll understand what I mean. This makes writing a simple crud app fast, but testing it sucks because you have 2 things tied together - your actual data models and your database layer. Funny that nobody would ever do that with the filesystem, but it's super common with the database.

To make your models clean and easy to test, you need to pull out the ORM bit into something more pluggable.

The same principle applies to writing testable biz logic. A lot of times people don't know where to put it so it ends up in the View, Controller, or Model. All of which are the wrong place usually. Testing that code becomes not awesome because you then have to write a test against what the view outputs, the controller does, or possibly what the model is doing (usually by touching the database).

The answer I've found is to pull your business logic into a separate container that can be tested easily apart from any MVC structure.

None of this means you need to over-abstract anything. It just means you have to figure out the right abstractions and structure for the right layers. None of this is by any means obvious at first.

P.S. I'm working on an architecture framework that simplifies a lot of this to make doing TDD much easier.

mpweiher4• 14 years ago

Odd, my experience is exactly the opposite: code that is written using TDD tends to be extremely concrete, because it does only as much as is needed to pass the test, no more.

If you are making code overly abstract, that does suggest you may be overthinking your TDD approach.

vegas• 14 years ago

He left out "You are Donald Knuth"

readme• 14 years ago

I'd be kind of surprised if Knuth didn't write tests. I'd expect him to formally prove every single line of his code.

In any event, I find that using literate programming results in less buggy and more comprehensive code.

mpweiher4• 14 years ago

Donald "beware of bugs in the above code; I have only proved it correct, not tried it" Knuth?

ssi1111• 14 years ago

I am a big fan of TDD. I read this post to see if I can get any reasonable arguments against TDD, so I can be better prepared next time I am trying to sell TDD. #10 - 'no clients' and #8 - 'short project' kinda make sense to not practice TDD, but these also mean that there is no real product - you build something and you throw it away after a few days. So there is no need for TDD, testing or anything really. All the other points in the post don't make any sense :)... is it just me?

mcs• 14 years ago

so silly. brogrammer turned maddox.

Reasons to Avoid Test Driven Development

Related Stories