How I Quickly Test and Validate Startup Ideas

http://startupbound.com/how-i-quickly-test-and-validate-startup-ideas/

by dberube • 15 years ago

205 74 15 years ago

startupbound.com

wildbunny• 15 years ago

Using adwords to provide enough traffic to test your product viability is genius...

I think the HN effect has taken their site down, though!

dberube10• 15 years ago

It has! It's at Rackspace and I'm working on getting it on the largest server they have.

Didn't expect to hit the front page.

tszming2• 15 years ago

Would you mind telling how much traffic (e.g. visits per hour from HN) you get when your article hit the front page?

TamDenholm• 15 years ago

Down for me, heres the google cache.

http://webcache.googleusercontent.com/search?sclient=psy-ab&...

egiva• 15 years ago

Actually, other people have done this in the past - Timothy Ferriss (the biz guy and author of "The Four Hour Work Week") was famous for using this adwords multivariate testing technique when he was deciding the future title for his book! See a great interview here: http://www.youtube.com/watch?v=3JHzLlvlVmQ

dberube10• 15 years ago

It's back up -- upgraded the server.

brador• 15 years ago

I don't know what you guys have been drinking last night but HN is full of absolute gold this morning. Great article.

mpunaskar• 15 years ago

@dberube

I have used same method to collect beta tester for one of my idea. Google adword has given amazing reults so far. Thats why i am interested to know if cost factor the only reason to choose microsoft adcenter over google adwords ?

Does it give more ad impressions for less price + better click through rates?

dberube10• 15 years ago

The quality of the people coming to the site, in my experience has been the same.

I actually have better CTR on adCenter compared to AdWords for the same ads on the same keywords (marginally better).

So it's really come down to cost for me. In my experiences it's consistently double or more for the top keywords in the bunch.

I've personally never run a massive campaign to drive tens of thousands of visitors to a site through either service. My guess is, if you need high volume of people quickly, you need AdWords.

wooyi• 15 years ago

Doctors don't buy software on the internet. You can't validate the idea that way. The sales cycle is long and their buying process is totally different from the "convert on the net" SaaS model. I did a startup called Maviq in 2008 for the patient reminder market. There is a v. high barrier to entry in this market for startups.

dberube10• 15 years ago

That was just an example, not the basis for the process.

vidarh• 15 years ago

While you may not be able to validate the idea of the service in isolation, you can however validate whether or not there's a market for a service like this sold over the internet.

In many cases, someone may consider it worth excluding an otherwise potentially profitable idea if the sales cycle is long and the buying process is different from the "convert on the net" SaaS model.

famousactress• 15 years ago

That's changing, frankly... and rather quickly. I speak as someone who builds (and sells) SaaS products to physicians.

rorrr• 15 years ago

There are many private practice small business out there.

wwdevries• 15 years ago

Can you define what you consider a "high conversion percentage" in this stage of testing?

mluiten• 15 years ago

I actually like to add an intermediary step between the landing page and the e-mail form: a "Create an account/Buy now for $X/month/year" button. When people click on it, they get a "Sorry, we're not done yet, but give us your email and you will be notified once we release" + the email form.

Not only do you know people are potentially willing to pay X dollars for your product, you can also do a multivariate test on different amounts or pricing schemes to see if they convert differently, giving you some rough idea how much your idea is worth to them. I always add a "Make an account for FREE" as well, as a baseline.

dberube10• 15 years ago

Well, the problem with doing that at this stage of the process is you risk getting the initial price completely wrong and nobody clicks it.

Even though, if you had priced it just $20 lower, everyone would have clicked it.

So by putting price in the mix before you even identify a willing market, you risk missing the ball completely.

cbr• 15 years ago

You could A/B on price to get the demand curve.

mdda• 15 years ago

You could put up a separate pricing page to see how many people click through to it, and then look at hovers to see where they linger... But that's obviously going a bit deeper than the original idea of slapping up a single page to gauge interest.

tagawa• 15 years ago

It's probably just me, but I can't help feeling it's a bit underhand to offer something for sale that doesn't exist yet (unless you make it clear upfront).

phzbOx• 15 years ago

Really interesting and well written. I particularly liked the part when the author said not to show feature as if they don't buy the silver bullet, they won't buy your product.

kamikazi2• 15 years ago

Fantastic article dberube, couldn't have come at a better time for me. Few queries: a) What would you call a good collection, conversion rate? (Also asked on yr blog) b) Will this work for ideas that might be seasonal? (eg: I am thinking of selling a particular fruit from India online - it's seasonal and the store will operate only abt 4-5 months. If it's successful, I'll see what else I can do with it, but as of now there are no plans). So, have you worked this strategy for any such ideas which are not 24x7? How early before the season do you think I should run this. My season is still 5 months down the line. If I run too early (now-ish) I run the risk of not getting anyone interested as no one is searching for it. c) Can you give a few figures from your 3 experiments about how many visitors you got from adCenter? And their conversion rates? Thanks.

helen842000• 15 years ago

You could always start by collecting e-mail addresses of those people interested to buy it.

So they sign up and get notification once you have the items ready to buy.

Consider your shipping methods carefully else this type of product might be tricky.

kamikazi2• 15 years ago

Well, that's my fear. That if I run this too early I may not be able to collect any prospect email addresses (as almost no one/very few might be interested in it or searching for it at this time).

I do have some queries/concerns on the shipping front too. Have you worked on this type of farm/agriculture product or know any pointers where I can get more info?

ps: Updated my email in my profile (if you want to take this offline). Don't see your email ID in yr profile.

chunkyslink• 15 years ago

Any good alternatives to unbounce?

dberube10• 15 years ago

It's the best one I've seen. There is also http://www.instapage.com/ but it's not as good and I've run into problems with pages not loading -- and when you're paying for traffic to hit a page, you want it to load.

cesarpereira• 15 years ago

http://www.kickofflabs.com/ is a good alternative as well. They basically sit between Unbounce and LaunchRock as far as features go.

mva• 15 years ago

I've read about this idea in the 4 hour work week. This is the way Tim Ferris tested his ideas.

matdwyer• 15 years ago

Agreed, the 4HWW covers this fairly well, along with some tips I hadn't thought about previously

powertower2• 15 years ago

I'm kind of biased against using Google AdWords since they banned my account for no explainable reason (they lack complete customer care to the point where they can't even display the ban message right: http://www.devside.net/images/adwords-account-suspended.png), but consider spending more money and getting better validation by going directly to websites specific to your market and buying ad space there.

The AdWords traffic is just too broad and "general public" and it's incredibly difficult because to get good CTR (click-through-rate), you actually have to find keywords that produce bad or irrelevant results for the searcher while being on topic, as otherwise the searcher will simply click the organic results and never even see your sponsored result (when was the last time you went to Google, did a search, got good organic results, and decided to skip that and just go to the sponsored link?).

AdCenter is a little better IMO, but niche websites are absolute killer since you're no longer screwing around, trying to filter out irrelevant traffic, trying to get the right keywords and bid prices, and making sure your landing pages have the right quality score. The moment you get a low CTR is the moment Google AdWords start increasing prices, stops showing your ads.

Go right to the source to test your ideas. It will take more money, but you'll get a confidence level of 95-99% for your validation, rather than something much much less that you'll get using AdWords. It will be quicker too; AdWords keywords can be incredibly difficult and time-consuming to get right.

suking2• 15 years ago

You have no idea what you're talking about with AW.

skulvr• 15 years ago

ADW is overrated and very expensive. Specially if you need an expensive keyword. I had a good experience with Facebook ads. It was cheap and high CTR.

felipe• 15 years ago

That happened to me too. I was testing an idea with AW and was banned for life, without having any chance to talk to a real person.

acangiano2• 15 years ago

I may be biased, as I'm part of the Google Engage for Agencies program, but for all of its defects, AdWords is an incredible marketing resource with an 80% reach.

Their two main problems are:

1) Customer care for end users is admittedly lousy.

2) You really, really, really need to know what you are doing or else it's very easy to be taken to the cleaners.

Ignoring AdWords as part of your online advertising efforts is usually a bad idea. However relying solely on it is a dangerous proposition, too (case in point, you were banned).

Diversify by introducing media buys (as you suggested), other ad channels (like AdCenter, LinkedIN ads, Facebook ads, StumbleUpon ads, etc), and then prioritize based on ROI.

The question is, do you need all this to test out an idea? Probably not. I think that at this stage virtually anything targeted will help you gauge interest.

> AdCenter is a little better IMO, but niche websites are absolute killer since you're no longer screwing around and trying to filter out irrelevant traffic.

Don't forget that with AdWords you can reach niche sites, and even target them individually if you want to, so you don't need to commit to paying $xxx/mo for a media buy directly with the publisher (just to run tests).

powertower2• 15 years ago

> Don't forget that with AdWords you can reach niche sites...

That is true, BUT, since AdSense brings in pennies for the webmaster in general, relative to the other options, a lot of good/non-shit niche sites are not using AdSense, but are either using custom sell space or other targeted networks that payout 2-10X of what Google AdSense does.

IMO, if you are serious about running and monetizing a business, you will get as far away from being 100% dependant on the Google index and their ad network as you can.

So many people have been screwed this way. Imagine what would happen if you dropped a couple of pages down in the index tomorrow, or your AdWords account was banned.

Though I'll still admit there are a couple of sites I can no longer target, because they use AdSense ... but I have considered emailing each one and offering 2 or 3-times whatever they get with AdSense to get rid of it and put up my ads/links instead.

eps• 15 years ago

> If everyone is busy, those calls are often not made.

I personally have not seen a single doctor's office that does NOT make reminder calls or forgets to make them. Moreover virtually all of them also have a $50 fee penalty for missed appointments (which is never charged, mind you).

In other words be careful if you are validatng the idea just for the sake of proving yourself right. Especially when your landing page is so ambiguous in its message, that it's basically detached from the original idea.

dberube10• 15 years ago

Once again, the example given in the post was not an actual test. I was using the idea as an example to guide the article.

The startup idea in the article isn't the focus.

fdb• 15 years ago

I've heard this called "pretotyping". The basic idea is not to create a working prototype (which still can take weeks/months/years) but to judge interest.

http://www.pretotyping.org/

markbao• 15 years ago

If you're looking for a fast landing page creator like the one mentioned in the article, I'm working on one: http://seekevidence.com

hypnotist• 15 years ago

Correct me if I'm wrong...

Are you using unbounce to create unbounce alternative? That is pretty badass.

dberube10• 15 years ago

Hahaha

jeffrese• 15 years ago

LaunchRock is a great landing page tool for testing a startup idea. I think they are a Y Combinator company too.

acangiano2• 15 years ago

But they are not open to the public yet.

freshfey• 15 years ago

You might be scrapping an idea off just because the landing page wasn't optimized - could this be a possible problem?

vsl22• 15 years ago

I don't know about having "silver bullet" advertisements be your landing page and text ad. It looks suspiciously like spam or some kind of con (think about the majority of "Work From Home" ads). Every reasonably smart person I know wouldn't trust "silver bullet" ads enough to seriously consider paying for the product being advertised. In the particular example, most doctors are supposedly smarter than the average population and so I think the conversion rate would be even lower.

I don't claim to know a better way to garner attention through advertisement than eye-catching (yet suspicious) claims, but I think the described method helps you determine the interest level in your product from a small subset (more trusting or naive, depending on your point of view) of your target audience, not your total target audience. Maybe there's a known "naive-to-total" extrapolator that I'm not aware of.

alabut2• 15 years ago

There's a balance to be struck because the polar opposite - discussing features rather than benefits - is the norm and it's worse than ineffective for the average user, it's confusing noise.

I just came back from running two days of usability tests on-site at a security software company. I wish everyone could see the test subjects' eyes glaze over the typical marketing text describing exactly how the thing works and conversely, how they'd light up about 20 minutes into it when I'd guide and moderate them towards the hidden nuggets of real tangible value, like "4x faster than Norton" or a chart showing that the memory usage was a tiny fraction of competitors.

vsl22• 15 years ago

I agree that there needs to be a balance and furthermore, that finding that right balance is very difficult. Probably why marketing whizzes get paid the big bucks.

In your example though, there is difference in that you're an actual person representing a reputable company on-site, not a random anyone-could-have-created-this advertisement on the anonymous web. From more trusted sources, I'd be interested in hearing the benefits and not necessarily the features. But in an environment where seemingly every third website activates my anti-virus software and I read about data hacks every week, I definitely don't trust Internet ads filled with hyperbole and exclamation points.

dpcan• 15 years ago

These landing page systems have a lot of room for improvement.

Upon getting a name and email address, it would be quite valuable to present the new subscriber with some form of survey. Be it short and multiple choice, but instead of JUST seeing if there is interest, you may be able to find out what aspects of your product bare the most importance.

ie If you create a landing page for a new shopping cart product, you may want to ask: Will you be selling digital goods? What country are you in? etc.

Vandy_Travis• 15 years ago

Agreed, 100%. I've actually bundled a Wordpress install with the necessary plugins and libs (google analytics, basic survey abilities, a/b testing harness). This makes it super easy for me to spin up an ec2 or linode instance. Then I can run a test and make a decision without touching any of my core products/sites/servers.

Not too difficult to do.

lesterbuck• 15 years ago

That sounds really interesting. Could you share some more details, or point us to an example that is running? Which plugins work well for you?

lbl3• 15 years ago

I wish you had written this a bit earlier :) I invented a playing card game and was really excited about it because I got good feedback during play testing. I tested it in few bars and they all liked the prototype. So after making some edits to the game rules from the play testing feedback, I went ahead and printed 2000 decks and I have a website where I'm selling the decks ( http://acardgamewithattitude.com )

I'm not seeing quite the success in selling online like the feedback I got offline.

No sure how I would have tested this concept before printing because its a niche market i'm targeting.

Now i'm confused what is wrong - Is it the site design - copy on the site - Am I over explaining - Is it not clear if its ia deck of physical cards

Can i take the site down and do a teaser like it is going to launch? and then sell it?

mrbgty• 15 years ago

I don't really know what I'm talking about but I'm not sure this type of card game translates well online. In my opinion, the site design is a problem. The problem I'm seeing is that there's nothing making me want to learn how to play a new card game. I have no idea what the answer to that problem is but there's got to be some motivator on the site, I think, to get people to want to learn it. I'm sure once the game is learned people will love it. Maybe you could look at techniques other card games/board games use to get people interested? If you made a digital version of the game, you may also be able to sell the physical card game online?

The site is too red.

lbl3• 15 years ago

thanks mrbgty for taking the time to look at the site and giving feedback. "Motivation" site is "Too red" I have added that to my list of to do. Thanks. Anyone else?

rokhayakebe• 15 years ago

I agree. I wouldn't buy from this site if they paid me double the cost.

Edit: If you want people to take you seriously, you should take yourself seriously. I am sure the game is great, given the offline response, but online we judge you by your website.

asmithmd1• 15 years ago

Clicking through to the authors blog home page and reading between the lines a little it looks like he is the founder and was the CTO of Mofuse.com and was kicked out just a week ago. Mofuse took $1.2M from 33 angels this past January so I guess they had something to say.

What is the story dberube? Inquiring minds want to know.

dberube10• 15 years ago

I demanded $40MM a year in compensation and the use of a private jet. I was told no.

drudru11• 15 years ago

I think this has been common practice for over 2 years now. Where did you learn this technique from?

yonasb• 15 years ago

Great post. I've used LaunchRock (widget, not the whole page setup), would definitely recommend it.

ww520• 15 years ago

I've visited LaunchRock to check it out. I didn't get any account after set up. Somehow I have a nagging feeling of being trolled, landing on the landing page of a site promised to create a landing page.

The FAQ says I will get an account after getting 3 people to sign up. Bahh. I just want to check out the features and I have to spend effort in marketing your site? Thanks but no thanks.

swatermasysk• 15 years ago

Warning...shameless self promotion...

Would love to hear your thoughts on my own start-up, Kickofflabs: http://www.kickofflabs.com

No gimmicks, tricks, etc. Just a solid product with fanatical support.

roythunder• 15 years ago

One reason I think the advice in this post is valuable is not just to test the startup idea, but also to test the founder's appetite for traffic acquisition.

Many technical founders like building products, but those products will fail without the majority of energy directed toward getting traffic. So if you can't get a couple hundred email addresses of interested customers, you can save yourself a lot of time by not going further.

ideaoverload• 15 years ago

Zynga's 'ghetto' testing is the same idea, maybe even more radical in details: http://grattisfaction.com/2010/01/how-zynga-does-customer-de...

inopinatus2• 15 years ago

This is a novel online take on the general concept of test marketing.

I would love to see a statistical analysis of the results.

markoma• 15 years ago

Just don't know how better to do this.. Apologize for breach of etiquette. Saw a post of yours about a month ago regarding Puppet vs Chef. Really impressed with your comments and wanted to dive deeper. Alas, the thread seems to have died and I cannot append or reply there. Is there any chance you would be willing to discuss that topic more? Many thanks in advance.

inopinatus2• 15 years ago

what would you like to know?

ping me, @inopinatus on tw*tter.

willpower101• 15 years ago

"If people like your startup idea then odds are you're not thinking broadly enough" -I can't remember who said this.

Cool idea though. Definitely works for immediate user solutions. Not sure it would work as well for innovations.

"People don't know what they want..." -SJ

jdee• 15 years ago

I'm interested in learning how people go about testing ideas that are difficult to define in the small amount of screen real estate that an ad offers, or dont have a clearly definable customer type.

How would one go about testing the waters for 'mass market' applications like Twitter, Google+ etc where the end customer is potentially everybody? I cant see how an ad like 'want to keep in touch with your friends?' would convert at all. Any insight anyone?

nikcub• 15 years ago

This is an interesting idea that has done the rounds a few times in the past few years, but it is also a bad idea in most cases.

First, if Google find out that you are advertising and collecting email addresses for a product that does not exist - they will ban you. PayPal will do the same if you are collecting any sort of payment information. There is just too much that can go wrong and it can be easily mixed up as being a scam or spam (which it sort of is, if you think about it).

You don't want to be banned from Google (or PayPal) - they carry that against your name forever, even if you later signup as a company or with a different account.

Second, this can only help you in a market where your potential customer knows what their problem is and knows to look for a solution. In so many cases of successful products this would not have worked. Not for Google, Twitter, Facebook, YouTube, Microsoft, iPhone, iPad or most any large and successful product.

It can also lead to false hope, or worse, an assumption that a product may not work. Twitter started out as a messaging system for cyclist couriers. If Jack had read Tim Ferris[1], took his idea to test the market with search engine ads, he may have come away unimpressed and believing there was no room for such a product - without even realizing the broader possibilities for such a messaging system.

It has become a bit cliche to quote Steve Jobs, but from his bio:

> At the end of the presentation someone asked wether he thought they should do some market research to see what customers wanted. "No," he replied, "because customers don't know what they want until we have shown them"

The better way to test the market is to actively market (as opposed to passively market - ie. having the market find you) - ie. go out and find your potential customers, the influencers in the market, analysts that cover the market, etc.

The whole idea behind minimal viable product is to lower the risk involved in testing a market - so get the MVP built and then promote it, but not just with search engine ads (the part of the market that knows what it wants, likely to be the smallest part) but also by actively reaching out.

The post suggests a budget of $20-50 a day just to get some leads, when with that sort of budget ($500-1000) you could just get a first version of the damn product built.

You can also find out how many people are searching by just using Trends or a similar product (keyword analyser). The PPC part just means you are paying $10 per email address - contacts that you may never end up using

I would like to see that similar posts to this one proposing this idea stress that results from search engine marketing are not the be-all and end-all. The language in this post suggests that the results of search engine marketing for a non-existant product conclude the viability of such a product:

> If you weren’t lucky enough to have a high conversion rate you can scrap the idea or rinse and repeat this process and tweak or pivot on the original idea.

This should say that your market may not be people searching for the product, not that you need to pivot the idea. This isn't how you validate startup ideas or products, it is how you validate part of a potential marketing strategy for a idea and product.

(Edit: and thinking about it, as a customer, I would hate this. If I were searching Google and comparing products that I need for a solution for and bumped into your product and gave my details, after your website suggested that the product exists - only to later find out that it * doesn't exist* - I would be very very pissed off about it. Especially if I needed a product right now and was falsely lead to believe that a solution exists).

[1] Tim Ferris is cited a lot as being one of the first to do SE testing (in '4 hour work week'), but I also remember Josh Kopelman writing a blog post about it a while before that as well. IIRC, Ferris suggests that you do this when you already have a product, to test potential new features, pricing ranges etc. The OP probably should have at least cited Ferris in his post, since this isn't exactly a groundbreaking idea and nor is it the first time it has appeared on HN.

How I Quickly Test and Validate Startup Ideas

Related Stories