When you shouldn't bother unit testing

http://invalid-arg.github.io/2013/05/16/when-you-shouldnt-test.html

by invalid_arg • 13 years ago

44 42 13 years ago

invalid-arg.github.io

smanuel• 13 years ago

The tl;dr part should just say "If you haven't started coding yet, then you shouldn't bother much about unit testing" which is.. well... common sense.

thirsteh2• 13 years ago

Then test-driven development defies common sense. Its whole point is to write a runnable specification in the form of a test suite before you write the actual code.

kleiba• 13 years ago

...which can have its benefits, but can also go terribly wrong if you get lost.

(Case in point, Peter Norvig comments on a famous series of TDD blog posts that had the goal of writing a Sudoku solver, but ultimately failed:

"And I could see, right from the start, you put these two together, and you could solve this Sudoku thing. He didn’t know that so he was sort of blundering in the dark even though all his code “worked” because he had all these test cases."

http://pindancing.blogspot.com.au/2009/09/sudoku-in-coders-a...)

norswap• 13 years ago

Isn't TDD all about writing tests before coding?

warmwaffles4• 13 years ago

I like MDD, money driven development. It usually works wonders ;)

I mix between TDD and not writing tests until after a section of the code is done. It really depends on what I'm trying to accomplish. If I'm spiking things, I tend to just write tests to confirm my work and make sure I'm sane. Other times I know exactly what I want and write the tests first then code. Really just depends on the situation.

jerf• 13 years ago

I'd caveat the "you don't know the framework well" by first taking a moment to establish that you can meaningfully test the framework. There's a whole whackload of frameworks out there of all different sorts, even very popular ones, that haven't given a single thought to testability (or only thought about it so late in the process it was too late to do anything about it), either of the framework code itself, or the user code using it, and generally this results in a framework that can't be tested with anything lower than integration testing. I consider this a major negative for a framework.

I can't rigorously prove this, but I suspect that also points at having a framework that is going to tend to gratuitously break your code frequently if you try to upgrade, because they'll have no tests for telling if they have. And neither do you.

nahname2• 13 years ago

I've found one simple rule works well for the teams I have been on. If you can afford to re-write the entire application from scratch (considering time and money) then you can afford to skip unit testing.

On a side note. You will never learn TDD from a video. If you want to learn TDD, get very familiar with testing in general, pick a domain you are comfortable with and then force yourself to write tests for everything before code. Repeat 2-3 times and you will be better at TDD than 80% of the people who are presenting on it.

seren• 13 years ago

Do you think it makes sense to try to learn TDD on a language you don't know ? So let's say I want to pick Go (as a hobby, not for a particular professional need in the short term), would it make sense to try doing some TDD on my first smallish applications ?

Noughmad• 13 years ago

Why not? It's nice to make a fresh start, as with a new language you will not fall into your old habits so easily.

nahname2• 13 years ago

I don't think there is anytime not to try to learn it. It will be harder the more unfamiliar you are with everything else (language, libraries, domain, etc...)

Chris_Newton2• 13 years ago

From the introduction in the article, about the merits of TDD:

As well as the simple fact that your code will be more reliable, it forces you to write your code in a way that is testable, which in turn will make it loosely coupled and modular.

One of the major drawbacks of many current unit testing tools is that in some contexts they can have exactly the opposite effect. If you expose implementation details through your modules’ interfaces that would otherwise be hidden, just in order to support your unit tests, then that makes your code less modular and potentially more tightly coupled.

It could be a direct exposure, for example by adding public methods to a class that allow tests to access otherwise private fields in languages like Java or C++. It could also be indirect exposure, for example using a dependency injection scheme to initialize everything so you can substitute test dummies for live code, but then using something at the top of the tree that essentially has to know everything about everything in your application in order to set up all the injected parts.

These things happen to a greater or lesser extent depending on many factors: which tools you have, how dynamic your language is, and more. However, I would challenge any claim that using something like TDD necessarily implies that your code will evolve a robust, modular design. Extensive unit testing can promote modularity in some ways, but it’s definitely a two-way street in most contexts today.

JulianWasTaken• 13 years ago

Michael Feathers has a good talk where he discusses this a bit:

http://vimeo.com/43659006

rst• 13 years ago

One trick I've used in Scala code is to put the test suite in the same package as the code under test, which gives the tests access to package-private stuff which are normally inaccessible to other code. That probably could work for Java as well. (And in C++, I guess you could declare the test suite as a "friend" of the code under test --- "C++, the language in which friends can access your private members!".)

I feel a bit funny about this, though; it does make the tests explicitly dependent on stuff that, to the rest of the code, is a hidden implementation detail. So, it's probably better to test only the public interface wherever that's feasible...

gbog• 13 years ago

> challenge any claim that using something like TDD necessarily implies that your code will evolve a robust, modular design.

Obviously there is not "necessity" here. You can indeed add more coupling with wrong tests. But this is a side effect of some excessive cargo cult about TDD and you should not throw the tool away because some overuse it.

The claim that writing tests helps you define and describe you'r code interface still hold, and strongly.

I usually code in Python and run pylint on my prose. When it tells me I shouldn't call a private method or attribute from outside, it is for me a warning sign:

- I may not be testing the proper behavior.

- If I feel I need to test it inside, this code nugget has too much responsibilities.

- It may be the case that this private thing is really public (I am a private-by-default coder), but this is rare.

When you don't have the time to refactor it properly, I admit that in some cases there are only bad choices: writing a long unatomic scenario testing, testing atomically inners, or not testing anything at all. The later is the easy path for short-sighted development, but adding some testing, even subpar, is much better, because at least it shows Someone Once New How To Roughly Run this part of the code base.

kasey_junk5• 13 years ago

Changing production code purely to support test code is most certainly a design smell and almost always indicates that incorrect abstractions are being used.

That said, writing good unit tests, doing TDD correctly, etc. are all skills that require experience. Just using TDD is no more likely to guarantee good code than any other software development process. But no process at all is almost always worse.

warmwaffles4• 13 years ago

> It could be a direct exposure, for example by adding public methods to a class that allow tests to access otherwise private fields in languages like Java or C++

I hardly test private methods for this very reason. The methods are private and are meant to be helpers. You can poke at it through the method that implements it but I tend to stay away from testing private methods.

mbell• 13 years ago

I use groovy for writing tests. Much better language for writing tests in general in my opinion but also has the benefit of ignoring 'private' so you can directly test your private methods in isolation without messing around with reflection.

To head off the 'only test your public method camp':

I often have bits of math or 'tricky stuff' in a private method, the public usages of that method may limit the input to the private method in some manor or map it in a way that offsets edge cases, i.e. an edge case for the public method may not exorcise the edge case in the private method. I'd much rather just test the full input range and edge cases of the private method directly than leave it up to the next person who comes along. The next person may need to tweak the application logic in the public method and then also figure out how to map the application logic inputs to ensure the edge cases of the private method are tested, seems less than efficient and also prone to being missed. It seems silly and fragile to expect that person to figure out the edge cases for the public method, all the edge cases for private methods and the map of public method input to edge case in private method(s).

tel• 13 years ago

This "layered" effect of interface hiding is natural in pure functional contexts. In Haskell you often see exposed "Internal" modules for just this reason---they're not the primary API, but they're valuable to have around.

It's harder to have these layered effects in other languages (in my experience) though often distinctions between a concrete and abstract (interface driven) API are a good first layer.

Chris_Newton2• 13 years ago

There is a lot of merit in allowing modules to offer multiple interfaces that clients can use as long as they explicitly request them. You can solve the multiple-levels-of-abstraction API problem that way, and you can solve production/test/debug access control problems that way too.

Unfortunately, most languages today have at best limited support for this idea. For example, plenty of languages with OOP tools offer both some sort of public/private distinction for access control and a concept of implementing one or more explicit interfaces, but these are usually treated as orthogonal features.

As long as you’re working in that kind of framework, or with more dynamic tools where you don’t necessarily have any concept of explicit interfaces at all, it’s going to be difficult to write unit tests without circumventing the normal access control and modularity mechanisms somehow.

textminer• 13 years ago

The argument against testing seems to be "the Spike", when you're learning a language/concept/framework or are testing out ideas. But startups (at least the two I've worked for the last two years) seem way more heavily invested in the spike than in the carefully-tested production environment. Personally speaking, it's even more exciting to work on.

It took me a good year and half to properly appreciate well-optimized, well-tested code in production, and enjoy the challenges associated there (which aren't trivial-- the neatest, ugliest NumPy script can be entirely useless at scale, or in a timeframe useful to users).

But I still personally feel flat when the knowledge gains associated with a Spike aren't as frequent. How have other developers and math-types dealt with that disappointing feeling of a plateau? Besides just changing jobs entirely, I mean.

jabbernotty• 13 years ago

> But I still personally feel flat when the knowledge gains associated with a Spike aren't as frequent. How have other developers and math-types dealt with that disappointing feeling of a plateau? Besides just changing jobs entirely, I mean.

I'm not entirely sure if I understand you, but it sounds like we are thinking about the same thing.

I've started a job working with C# about 8 months ago. It is a comfortable language to work with. But this job, it's problems (the ones we solve) and C# haven't really exposed me to anything new, which leaves me feeling unfulfilled.

Is that what you are talking about?

falsedan• 13 years ago

Kind of hoping that the linked story was a continuation of Ovid's thoughts on (not unit-)testing:

http://blogs.perl.org/users/ovid/2013/04/how-to-be-agile-wit... http://blogs.perl.org/users/ovid/2013/04/when-must-you-test-... http://blogs.perl.org/users/ovid/2013/04/code-evolution-vers...

papsosouid6• 13 years ago

I wish TDD advocates would stop equating unit testing with TDD. Just because I am not a member of your wacky cult, doesn't mean I don't use unit tests. This article isn't about when to skip tests, it is about when to skip TDD. Big difference (and the real answer is always).

Roboprog• 13 years ago

I like automated tests when they are low hanging fruit. Most often though, I'll settle for specs that are detailed enough to tell how to operate a feature, rather than just a hand-wave. I'm not too happy when coworkers commit code and don't bother to run it at all :-(

See the comment somebody made elsewhere about "money driven development". The cost-benefit ratio has to make sense.

kasey_junk5• 13 years ago

TDD is a design tool nothing more. It happens to be a design tool that creates unit tests as a nice side benefit.

If you have a better design tool in your arsenal, feel free to use it, but before you slam TDD as a cult, I'd ask what is your design process and why/how is it better than TDD?

papsosouid6• 13 years ago

I am not slamming it, it is a cult. Read what he wrote, these people literally think you either do TDD, or you are wrong. Despite all the actual testing of TDD having found it offers no benefits at all.

prepend• 13 years ago

TDD has some challenges, but it addresses the critical need of having solid unit tests for code that allow projects to be modified and used by diverse sets of programmers who aren't going to reach out and ask you if their change broke your code.

I've frequently heard criticism of TDD most loudly by those who don't write unit tests, do unit tests manually, or intend to write them "after they get everything working." This last excuse is most egregious because one rarely has time to go back and write tests for something that is working because there will always be something more pressing than addressing the minor technical debt of writing "unneeded tests."

papsosouid6• 13 years ago

>but it addresses the critical need of having solid unit tests

No, it is entirely orthogonal to having unit tests, that's exactly what I was saying. There is absolutely no evidence to support the claims that TDD is more effective than just being a normal developer who writes tests as needed.

>This last excuse is most egregious because one rarely has time to go back and write tests

Says who? I think it is pretty absurd to invent fake problems to try to justify something. If TDD were actually useful, proponents wouldn't need to compare TDD with "don't write tests at all because obviously the only way to end up with tests is using TDD".

languagehacker• 13 years ago

Totally disagree with these assertions. You _should_ know these things, but you need to learn them by doing. You'll never learn this stuff if you don't do it.

When you shouldn't bother unit testing

Related Stories