Is the popularity of unit tests waning?

http://binstock.blogspot.com/2008/05/is-popularity-of-unit-tests-waning.html

by edw519 • 18 years ago

13 18 18 years ago

binstock.blogspot.com

anr• 18 years ago

We're always looking for a Silver Bullet. Does someone know if there are papers out there that correlate unit testing with better results in software projects?

jzell• 18 years ago

Even if there were papers on it, I think there are too many confounding factors to elicit anything useful from them. Personally though, unit testing and test first development help me focus on a small problem and solving that, while ignoring the rest of the system. It's really easy to get pulled into the big picture. I also found my code to be more modular and decoupled.

mattjung• 18 years ago

I am not sure that the failure of commercial products, frameworks and books on unit-testing are a sign for the decline of the usage of unit-tests in general. In our company, we are extensively using unit-tests, but use no books neither commercial products (only the junit-framework). Maybe unit-tests have just become common programming practice without the need for any more hype?

dgabriel• 18 years ago

I looked into Agitar's JUnit Factory, but had to turn away from it when I realized that you send your source code to them to generate the tests. I have yet to work for a company that would feel comfortable doing that.

stcredzero• 18 years ago

Unit Testing works great in Smalltalk. Refactoring Browser handily refactors your tests at the same time you're refactoring your code. I've heard complaints from people using other languages that Unit Tests make refactoring twice the work. There is no excuse for this. If your Unit Tests are just code, then the tools that change your code should just as easily change the tests that use the code.

menloparkbum2• 18 years ago

Does it matter? The guys who made Twitter are unit testing fanatics and it doesn't seem to have helped their situation.

icey• 18 years ago

Unit testing is not load testing.

menloparkbum2• 18 years ago

I understand that. It was a serious question, though. How worthwhile are extensive unit tests if your end product is still broken?

gruseom• 18 years ago

I don't see it waning. I do see it dissolving more into general programming practice, as opposed to being a big deal in and of itself. What does seem to be waning is the evangelical fervor surrounding it, which seems like a good thing to me (though not to the author). Like other fads in software, unit testing was oversold.

Meanwhile, the logic of the article seems weak:

1. A company that claimed to auto-generate unit tests for your code failed? Give me a break. You can't write good tests without understanding what you're testing. If it could do that, why didn't it just auto-generate all your code in the first place? One needn't look far to see why this failed.

2. Few OSS tools have been adopted (except xUnit which is used massively)? This is normal. Most open source projects fail, a few succeed big. xUnit won. You don't need much else.

3. Not everyone teaches it? Why should everyone do anything?

4. Not many new unit testing books? There are many such books already. What don't they cover? Why should there be a whole book about JUnit 4?

5. Alternative unit-testing frameworks aren't used... see #2.

The author seems to think of unit testing as if it were a domain. It isn't, it's a technique. Why should there be an industry around it?

ardit33• 18 years ago

Unit testing hast it's own use, but I never really drunk the kool-aid.

In my experience with unit testing:

1. It doesn't help much if you are working in client side enginering. No matter how clever your unit test is, it can't help on detected ui rendering errors, animations, or multithreading programming.

2. Java + unit test, means you have to throw away all encapsulation, and good OO design to apease your unit tests, exposing your program to your coworkers to create havock by calling methods that shouldn't be visible to them.

3.Unit tests can give you a false sense of security. Oh, look, it passed the unit tests, my code must be good. Well, most of the time unit tests != real world scenarios.

4. TDD (test driven design) of software, can be slow. I would recommend it to all my competitors. In a startup, I'd rather release something fast, that might be still little buggy, than something that works well, but it took forever to get there. Plus TDD is constraining, as you are programming around your tests, and not around what the user will see. I found often, that my final design on a functionality, is different from how I thought it would be at the beginning. When working in large systems, you don't have the luxury to know how your final design will be.

5. Unit testing stiffles creativity in design, goes against "Exploratory programming". If you already know the answer on what you are building when you start, then you probably are not exploring around.

6. Unit testing != fun. Honestly, I'd rather be implementing features, than testing them.

Where unit testing is good:

1. When building an api that will be used by other people amd that is more static, less fluid. As oposed to UI enginering, something like geoCoding api's, have simple and clear results. That is very easy to unit test, as the value returned by the api is within a determined range, and the unit tests can be few, simple, and clear.

2. Scripting languages. Unit testing helps on detecting errors, where in a static language they would have been caught earlier. Plus it is much more easier to unit test on something like Python, then something like Java, or god forbid C/C++.

3. Coorporate enviroment. If you are waiting for something form a different group/department, you can build some unit tests to test if this group's code is meeting minimum standards.

Overrall Unit testing has its place, but be careful not to rely on it too much. It can't be substitue to real testing.

bkovitz3• 18 years ago

I drank the Kool-Aid and I still like the flavor. I still practice TDD every day. And yet I agree with most of what you said.

When people get dogmatic about unit tests, they write way more than is needed to bring value. The value is: you can refactor code more easily; you can think things through a little bit at a time and let the design evolve; the code you write tends to be really simple and easy to understand; you get executable documentation of code as a side-effect. And for me, a huge benefit is that when pair-programming, the current test defines the current task.

If the unit tests are making the code harder to refactor and evolve, then it's time to write fewer of them. And if unit testing isn't fun, then something is seriously wrong.

Do people not know that when you want to just explore a wild design idea, you should abandon unit testing and just go wild?

jan_g• 18 years ago

I have found out that unit testing is great if you use it sparingly, that is only use it on public methods of public API. So, if I write server-side software, I unit-test only facade methods that are invoked by clients. And I always run the tests before committing new version, which is a pretty good safeguard.

It has saved me some headaches - especially when software grows in size and you simply cannot predict what impact even a simple change can have.

Anyway, I wanted to say that I agree with most of what you said :-)

herdrick• 18 years ago

Let's hope.

twak• 18 years ago

We don't use unit testing because 1) it kills agility. We're small, new & rewrite our software base every year. Complete code coverage slows us down by a factor of two and doesn't give that time back (so far) in reduced dev time. 2) we have a graphical application that's hard to unit test. 3) it's shooting bugs in a barrel & assuming that because you don't hit a bug there aren't any.

jarnold• 18 years ago

It takes as long to master TDD as it does to master a new language. I didn't get good until I had been doing it for a few years. @ardit33 is dead-on about the coolaid analogy. You either get it and can't imagine programming any other way, or there are significant roadblocks you don't want to overcome (UI) / just can't TDD fast enough to justify the time.

I haven't hit a point where I've invested too much into the quality of the software. It's fine with me if TDD isn't popular. I'll just keep using it as a tool in my arsenal to beat the competition.

Is the popularity of unit tests waning?

Related Stories