Ask HN: Should you TDD a MVP?

https://news.ycombinator.com/item?id=4986665

by fam • 14 years ago

57 36 14 years ago

news.ycombinator.com

Is it beneficial to do so or does it simply add too much overhead?

biesnecker• 14 years ago

No, if your MVP is a total throwaway that you're using solely to validate a hypothesis, and once/if it's validated you'll be redoing everything.

Yes otherwise.

My experience with TDD is that the cost/benefit curve doesn't really pay off until you need to refactor or significantly expand a project, at which point having good test coverage saves you an enormous amount of time/pain.

rhizome31• 14 years ago

My experience is that doing TDD doesn't take more time, even for initial development. It doesn't take less time either. It's subjective and I haven't done any measurement (I suppose it would be really hard to measure the effect of TDD, all other things being equal, since writing the same software twice using both methods would obviously be misleading).

I do have one clue that TDD takes the same time as non-TDD for me : I used to work as a freelance developer, paid by the hour, for clients to whom I didn't feel necessary to tell whether I would do TDD or not when negotiating the number of hours a given project or task would take. Then I was doing TDD and was still able to deliver on budget. Some of those clients were themselves developers who presumably would know how much time a task should take. The fact that I was doing TDD, or even writing tests at all wasn't part of the discussion about the amount of time it should take. Doing TDD didn't seem to have any effect on my ability to meet the schedule.

I think the reason it works like this for me is that I tend to do a lot more manual testing when I don't write tests (and if I don't write tests first, I tend to not write them at all because the motivation is much lower, the code is already "working" after all...)

jmount3• 14 years ago

"In fact I usually like to TYZCMZQ when I'm not VJBDTLJX-ing!"

jmount3• 14 years ago

Okay now the less sarcastic- but equally harsh answer. If you think test driven development is not good you should not use it. If you think test driven development is good and you don't use it on your minimally viable product: then you are signaling that even you don't find the product interesting enough to even automatically exercise.

ZeroMinx2• 14 years ago

I disagree. I like TDD, but if you're quickly building a MVP that has beforehand been decided(1) to be thrown away once you've gotten funding, just hack it like you've never hacked something before. You're aiming to prove a concept, to visually display your idea.

(1 - do make sure the non-technical people in the group really really really understand this)

paulbjensen• 14 years ago

I think it's ok to go without tests at the start of your project, but at some point you have to transition to using TDD, otherwise the ability to refactor the codebase or change existing features becomes difficult and time-consuming.

To give a rough idea, I spent ~ 2 1/2 months working on my app (Dashku.com) before I wrote a single test for it. Then when a company approached and asked to use it with a special requirement, I realised it needed tests in order to be deemed production-worthy. Now the app has tests (both unit and integration), and I'm now resuming adding features to the product.

schmrz• 14 years ago

I think that this is a wrong question to ask. You have probably so many more important questions to answer while working on your MVP.

If you are accustomed to test driven development then you will know (or at least you will be able to reason about it) when to apply it and when not to. It is just another development practice. But I do not think that you will fail if you do TDD or if you do not. But you do risk failing if you emphasize the wrong things such us should we use X or Y development practice. Use whatever works for you and your team (if you are not alone).

To give you a more concrete answer: follow common sense and your team's suggestions. Avoid making a development practice one of your priorities.

ollysb• 14 years ago

When I'm starting out I find writing a cucumber scenario is a great way to focus on getting a useful feature done. It's the first thing I add to a project. The general setup is done as part of getting that first scenario to pass. After that I do whatever is fastest. I use cucumber scenarios as tracer bullets for new areas of functionality but if I'm refining perhaps I won't bother. As always though, it depends...

sprobertson• 14 years ago

If TDD isn't something you normally do, it'll likely be too much overhead. If you're a TDD expert and that's what you always do, go ahead.

The purpose of an MVP is to ship something that works as quickly as possible, so it's not the ideal time to be trying out a new technique.

jdlshore• 14 years ago

This is a really good and interesting question, and it's one I've been struggling with myself.

The problem boils down to this: TDD makes your software more maintainable (if you do it well) and it lowers your cost of development. However, it also takes significant time and effort to figure out how to test-drive a technology for the first time. Everybody can TDD a Stack class; TDD'ing a database, or a web server, or JavaScript [0] is a lot harder.

So the answer seems simple: use TDD for the parts you already know how to TDD.

But it's not so simple! It's much harder to add tests to existing code than it is to TDD it from scratch. Sometimes, it's flat-out impossible. The expense is so high, there's a very good chance that you'll never get around to adding tests to the un-TDD'd code. It will hang around causing bugs, preventing refactoring, and sapping your agility forever, or until you rewrite... and a rewrite of any significance will halt your company in its tracks, so you won't do that.

So the reality is that, anything you don't TDD from the beginning, you'll probably never be able to TDD. Companies that go down this road find themselves doing a major rewrite several years down the road, and that's crippling [1].

There's another wrinkle on top of this: manually testing code and fixing bugs is expensive. Once your codebase gets above a certain size--about six developer-weeks of effort, let's say--the cost to manually test everything exceeds the cost to TDD it. (The six weeks number is a guess. Some people argue it's less than that.)

So the real answer is a bit more nuanced:

1. If your MVP is truly a throw-away product that will take less than six weeks to build and deploy and you'll never build on it after that, use TDD only where it makes you immediately faster.

2. If your MVP is the basis of a long-lived product, use TDD for the parts you know how to TDD and don't do the parts you don't know how to TDD. Be creative about cutting scope. If you must do something you don't know how to TDD, figure it out and TDD it.

3. It's okay to be a bit sloppy about TDD'ing the edges of your app that are easily rewritten or isolated in modules. But be very careful about the core of your system.

That's my opinion based on 13 years of doing this stuff, including building five successively-less-minimal MVPs over the last nine months for my JS screencast. The first three MVPs were zero coding, the fourth was a throw-away site, and the fifth was TDD'd with aggressive scope cutting to minimize the number of technologies that had to be TDD'd.

[0] Shameless plug: I have a screencast on TDD'ing JavaScript that I'll be opening up to the public, on a subscription basis, in a month or so. http://www.letscodejavascript.com .

[1] Rewrites are crippling: See Joel Spolsky's "Things You Should Never Do, Part I." http://www.joelonsoftware.com/articles/fog0000000069.html (There is no Part II, by the way.)

jt2190• 14 years ago

I completely agree, and let me add to your point:

> manually testing code and fixing bugs is expensive

The codebase almost always starts small and simple, and the ratio of cost of development to the cost of testing will look misleadingly acceptable. Here's a made-up ratio:

2:1

As the codebase and complexity increase, keeping those ratios in the same proportion when manually testing is pretty much impossible, and the ratio may actually invert. Again, a made-up number:

1:2

If the business is relying on rolling out new features to continue to grow the business, this could become a big problem if revenue isn't growing at an even faster pace.

sevenadrian2• 14 years ago

I think this is the comment I agree with the most so far.

I believe TDD is important, especially for a MVP one could argue. Do MVPs change a lot? Yes, obviously because you are learning and adapting to the needs of you customers. This is where have good test coverage (thanks to TDD) is a big asset. You can refactor, change, pivot, all you want, without the worry that removing this feature, tweaking that feature, or adding this other feature, won't break something else.

And manual testing is way more expensive then investing in automated testing.

In my opinion, unless it is a very short lived project that you don't plan to iterate on and isn't very important, you should have good test coverage. And good test coverage is best achieved with TDD.

This is of course just my 2 coins

wcoenen• 14 years ago

If you do decide to add tests to an existing code base, I found "Working Effectively with Legacy Code"[1] to be a good guide. Check out the table of contents.

[1] http://www.amazon.com/Working-Effectively-Legacy-Michael-Fea...

peterhunt• 14 years ago

TDD is good for some things, terrible for others.

TDD's great when what you're working on is easy to test, like your data backend for your application.

It'll vary based on your product of course, but in my experience I've found the frontend to be the big time sink in development. Using TDD for contemporary frontend development is, in my opinion, a big time sink since the cost/benefit ratio of frontend testing today is way too high.

tl;dr use TDD for your backend (data model invariants) and hack your frontend as quickly as you can (on top of a decent frontend stack so you don't get that messy).

mitchi• 14 years ago

I initially thought the question was "Do we ask our MVP to use TDD ?" -.-

christopherslee• 14 years ago

The key question here is, are you building a quick and dirty prototype, or are you building an MVP? MVP doesn't mean a garbage prototype. Depending on what you're doing, an MVP could still take significant time to build (and your MVP would take 1 month vs 3 months for the "full product.")

IMHO, if you're worried about supporting and maintaining your app, and concerned about your users not having a buggy experience, you should TDD it.

DanielBMarkham• 14 years ago

The question here is really "what's the test?" You have to reailze the MVP _is_ the test.

For a startup, customers are how you pass the test. Anything else is a red light. So in the most important way possible, as long as you have no customers, you have a test which is failing.

This is important because the maintainability you're looking for in a startup is your relationship with the customer. Manage that and the rest takes care of itself. If you are already in a business, yes, "maintainability" means writing code that will last. But if you've just got an idea or a dream, you've got nothing worth maintaining. Nor will you ever.

Put differently, your technical debt can never exceed the economic value of your code, which in a startup is extremely likely to be zero. (Different scenario entirely for project-based work for ongoing businesses, which is why TDD makes so much sense in that scenario)

sandis• 14 years ago

> "technical debt can never exceed the economic value of your code, which in a startup is extremely likely to be zero"

That's something to quote right there.

freshhawk• 14 years ago

> Manage that and the rest takes care of itself

Not quite, you are committing yourself to throwing away your MVP and rewriting once/if you find the fit. That might be a very good idea if your MVP is extremely simple and you expect it to either fail or succeed with minimal iterations.

> Put differently, your technical debt can never exceed the economic value of your code, which in a startup is extremely likely to be zero

Excellent point, but you probably want to factor in opportunity costs as well. You do not want to take on so much debt that it becomes impossible to succeed.

As soon as you start to iterate on the idea then the maintainability of your codebase starts to matter because that determines how quickly you can iterate on new ideas.

You do not want to get stuck doing even a partial rewrite while your competitors keep iterating their original higher quality codebase.

The overhead that testing or even full TDD add is very minimal if you have experience, so that's a tradeoff that must be made carefully. If you are thinking about learning TDD or testing while doing your MVP then that's a different story.

This is the same question as "should I use version control for my MVP" isn't it? The level of complexity for an MVP where this becomes a bad idea is different but it's the same question.

TheMonarch• 14 years ago

I agree, the assertion that you shouldn't test is absurd. For example, TDD can save you time by catching bugs for you. When Iv'e worked on apps without tests, I have to test manually each time I make a build, which gets annoying. That's time I could spend coding. I think the solution is something in the middle ground. TDD can sometimes veer off into a time wasting effort for the sake of TDD. Just TDD the complex parts, or the ones that are likely to break. Also write the tests that are easy, or quick to write.

LnxPrgr3• 14 years ago

Personally I wouldn't, but my opinion doesn't matter.

Does TDD help you? If so, use it! If not, don't! The whole point of an MVP is to ship something ASAP. Do whatever helps you do that.

Personally I've found TDD to be a tremendous waste of time for most things, but I know some people swear by it. It's your product. Git 'er done in whatever way you've learned to get things done.

Edited to add: People talk a lot about TDD leading to more maintainable, less coupled code. There's more than one way to skin a cat—not using TDD doesn't necessarily doom your codebase, and using TDD won't necessarily save it. Like I said, do whatever works for you.

j45• 14 years ago

What I'm writing below is meant in general, and not at the OP.

The purpose of an MVP is to find the minimum amount of value that a customer can't live without. An MVP is finding the 5 things you should have done in hindsight out of the 30 on your mind.

It sounds like you're focussing on yourself, and making your own life easier with TDD and not the customer. Wrong priority. You shouldn't expect your MVP to be here in 10 months.

Quit trying to prematurely optimize an empire that doesn't exist yet. Even if you write everything perfectly, you'll rewrite it from coding yourself into a corner and having to work out of it. You don't know what the problem-solution fit is, let alone whether theres a market of people willing to pay for it.

As history has shown, customers don't care what you code in, or how you code. It's more about building software that solves a pain and a problem, first, if you're serious about using the term "MVP". Sometimes folks keep doing what they've been doing and sprinkle some lean terms on it to make it sound like they're different.

TDD helps where there is complex business logic, not getting and setting basic information, which most MVP's do. If you have something that complex, it might not be an MVP.

Focus on the MVP, finding a product-market fit, and a problem-solution fit. Get out of the building and quit using methodologies as an excuse to analysis paralysis yourself.

davidw• 14 years ago

You should at least put a testing framework in place, and test some low-hanging fruit. That'll make it easier to get momentum going for more testing if things continue, and at least give you some minimal coverage.

stakent• 14 years ago

And test at least fundamental parts of user interaction with the MVP: sign up, login/logoff, conversion funnel, what else depends of MVP. MVP may have limited set of features, but implemented ones should work.

This not TDD but rather BDD without unnecessary fluff.

Additionally manual testing is repetitive work. It is better to outsource it to the computer to make sure that testing is in fact done when needed - preferably after each code change. By doing testing this way is far easier to isolate immediate cause of failing test - developer knows what was changed recently.

rhizome• 14 years ago

And just as an anecdote, I built a prototype for a client that laid fallow for a couple of years while he worked on some parallel business ideas that I also built, but when it came time to return to fire up the original idea there had been major language and framework version upgrades in the meantime. I had to almost rebuild the app from scratch due to the lack of tests in the MVP that could tell me what was what. Basic tests would have cut the upgrade time in half. Not that I mind the billings, but this time and effort gets in the way of the bizdev.

volandovengo• 14 years ago

short answer: no.

The purpose of a MVP is to prove the idea using the minimal amount of effort. TDD takes about 50% more time.

martinced• 14 years ago

Seen that TDD refer to "Test Driven Development" I was nearly sure that the question asked about "MVP" / Model View Presentation.

I've implemented lots of different of UIs using MVC type 1 and 2, HMVC / PAC etc. and, as a developer, I don't have single clue as to what "MVP" means here. Maybe some context to remove the ambiguity would be good in laconic questions like this one, especially seen that the other acronym is used.

Google for "TDD test driven development MVC" and you'll see that I'm 100% correct on this one ; )

sevenadrian2• 14 years ago

I believe by MVP he means "minimum viable product" http://en.wikipedia.org/wiki/Minimum_viable_product

PaulKeeble• 14 years ago

Those that TDD a lot have come to realise what TDD does is help you find a good design. It teaches you something about the way in which you code and design and allows you to cheaply play with the alternatives. It also ensures the code works and ensures so in the future as you undergo changes.

BDD for functional testing ensures that the functionality you test works and again that before you cut code you have thought about how your functionality is going to work. These are valuable exercises to do with or without the code to run.

So when it comes to an MVP to test the idea how well does it have to work and how well does it have to be designed? Some amount of your system will be critical and if in the process of changing it you break that functionality you may have no product. A certain amount of confidence in the core of what you are building is important, having it automated will allow you to change quickly. Ensuring 100% code coverage is not important, some bugs when you are small are probably acceptable.

programminggeek• 14 years ago

I would TDD in either case simply because if you end up maintaining it at all, you're going to wish you had tests. Also, I wrote an architecture that makes starting from day one using TDD much easier. It's called Obvious: http://obvious.retromocha.com

Once you understand the concepts, you can whip up a simple crud app in a couple hours. Example app here: https://github.com/RetroMocha/obvious_status

pbiggar• 14 years ago

I'm the founder of https://circleci.com, so I feel I have a little perspective here from talked to dozens of startups about this very question.

You should test a different amount based on your goals. TDD does not deliver value to your customer directly, so the only way its even usable in an MVP is helping you iterate and deliver the MVP faster.

A common mistake, and it feels like you're doing it here, is to think that TDD is a best practice that you have to do, rather than thinking about how to apply it to improve your productivity.

So how will testing help in an MVP? It allows you verify that you haven't broken anything when you write new code, and when you rewrite existing code. But because you'll constantly throw out code, spending a long time doing this will literally be wasted as you throw out the code you're testing.

Avoid writing the sort of massive extensive tests that many associate with TDD. When you unit test a function, test or two cases at most, any more that that is overkill. If it takes you 2 minutes to write a function, a function + tests should take at most an extra 30 seconds. If you can't think how to test it, don't.

So some specific advice:

- when you write a function, you're likely to test it somehow manually (refreshing the browser, at the REPL, etc). Take that manual test and put it into the test framework. It should take 10 seconds more. If it is more effort than this, don't bother.

- write MVP-type tests: "what's the minimum I can write here to viably test this"? 1-2 test cases, at most. A positive test and a negative test ideally. And then you're done, move on.

- avoid full-browser tests. Those things are a massive time sink, and nothing will change more than your front-end.

- use a CI service (self-plug: https://circleci.com). This means you don't have to run tests before you push. Just push and let the server catch it.

frio• 14 years ago

It really depends on the degree to which you do it. I'd argue that just having one or two very basic acceptance tests is probably all you want when you're testing the waters with an MVP. The most all-encompassing test you can make; and make it from a client-side perspective (so: JS or Python or something exercising your HTTP API).

For instance, if you were making a library app: test that a logged in user can add a book to it. That's as simple as:

    class AcceptanceTest(TestCase):
        def testThatAUserCanAddABook(self):
            r = requests.post("/books/", data={"title": "My New Book"}, auth=HTTPBasicAuth('user', 'pass'))
            self.assertEquals(r.ok, True)

Then, do the absolute bare, hacky minimum to get that test passing: implement a server which simply returns 200 to that URL.

As you're working, I've found that having that singular, overarching acceptance test will speed you up massively; there's no need to arduously step through your UI, or run through various httpie commands; the test covers you as you refactor to implement your user authentication, basic API, and backing database.

So, yes: TDD can be a big time investment, and can slow down your launch if you fully break your work down into unit tests. But having simple, straightforward, all-encompassing acceptance tests can give you a great base for getting work done.

(Note that after launch, once you start writing more tests, you should eventually throw away that acceptance test, as more componentised and faster unit tests replace it :)).

vlokshin• 14 years ago

front end, no. It changes too much. Back end, maybe. Only for the core logic that (probably) won't change.

Basic rule: don't let work create more work if the benefit's not worth it. You probably knew that already, so just stick to it.

gkarthik• 14 years ago

It depends. A MVP is basically used to test if your core functionality is popular among users and also to collect some useful feedback from users via a simple system.

I personally never developed a MVP for my first start up, Tune Patrol.

While, while making Tune Patrol, a social music discovery platform, my MVP would've involved just putting up a few songs and letting users play them. Like I said, this would've helped in two ways,

1. Get traction. Users know that you're up and running.

2. It helps test to see what kind of features a user might want by using a good feedback system.

Instead I went on to develop a product with a lot of functionality. My first problem was that I couldn't immediately determine if users liked my understood the functionality completely. Luckily for me turned out users understood pretty much everything and now we are developing on it.

You could also read "Lean Startup" by Eric Ries where he stresses upon the importance of a MVP and quotes a few examples from his own experiences.

Ask HN: Should you TDD a MVP?

Related Stories