I Eliminated the Free Plan from my Web App for a Month: Here’s What Happened.

This is a good idea to try, but November and December are weird months and it might not be reasonable in your space to assume they are comparable. I work in comparison shopping where it's really not true. You mentioned A/B testing, and I think you should try this again as an A/B test so that you can eliminate seasonality.

dmix3• 16 years ago

Why are November and December weird months?

requinot59• 16 years ago

holidays, christmas & new eve

Khroma• 16 years ago

I noticed that you changed your colors to a lighter color.

From what I see, many logos can be modernized so easily by switching to "pastel colors", or colors with more white in them. These are duller, but in my opinion, much more modern.

yesimahuman3• 16 years ago

Just curious, what is your reasoning behind requiring a username and an email? I've been experimenting lately with super minimal signups and the two seem more and more redundant for most web apps, but I want to hear your thoughts (and maybe it's a potential optimization point?).

matt15• 16 years ago

It was a part of the restful authentication plugin I originally used to set it up [1] :)

For this app its necessary because I allow users to share the timelines they create and I display their user names on the shared timelines. Your train of thought is good though.

[1] https://github.com/technoweenie/restful-authentication

yesimahuman3• 16 years ago

Yea public facing stuff seems like the only time it's really needed. One way is maybe to defer that until they actually share something and then have to pick. I sometimes take their username from the email (before the '@') and use that if necessary but it's not a great solution.

Good luck!

kevin_morrill2• 16 years ago

We just did phone number + password only for our SMS app textweight.com and the sign up rate is pretty amazing. I need to do some A/B testing with requiring e-mail.

yesimahuman3• 16 years ago

Crazy thought: phone number only, then after they are in, require they enter one? If they don't set one or forget, they just get a text and reset it? The psychology of this is interesting because on one hand they might feel less safe but on the other they might not care and just want to jump right in!

revorad• 16 years ago

I'm pleasantly surprised to see a good percentage of your free users upgrading to the paid version. Why don't you test a time-bound free trial before switching back to freemium?

matt15• 16 years ago

Great idea -- I'll consider it and may give a try in the future.

StavrosK• 16 years ago

Rather than doing this, I opted for a resource-based trial for historious. Free users get 300 bookmarks, if that's enough for them, that's great, if not, they can pay to get more. This way they don't put very much strain on the service but can still use it.

Another good idea, if your app can work with it, is to not let free users create any data. For example, for a website screenshot service, you could let free users access screenshots that paid users have created, but not create any of their own. This makes free users almost free to host but doesn't work for many apps.

dmix3• 16 years ago

I would be interested in a experiment comparing resource based limitation vs a time-based trial.

For ex. "upload 20 screenshots before paying" compared to "try the app for 14-days before paying".

kevin_morrill2• 16 years ago

Thanks for writing up this analysis and sharing.

idlewords• 16 years ago

I think all you've discovered here is that users find it annoying to be taken through a bunch of signup screens before finding out a website is not free.

You've drawn a lot of conclusions out of a very small experiment with a lot of uncontrolled variables. I'm glad to hear you're thinking of testing again more carefully.

dmix3• 16 years ago

If you have a paywall with no-trial, you must very clearly demonstrate the value proposition on the sales website.

Hiding the price until after the user enters their email is also bad UX. Not everyone is going to the pricing page before clicking sign up.

swombat3• 16 years ago

That's a terrible analysis. I don't see how you can come to those conclusions at all from the way you conducted this "test"...

My deconstruction of this "analysis"'s mistakes here: http://swombat.com/2011/1/2/data-is-dangerous

sjtgraham2• 16 years ago

Your points are valid, although I fail to understand why you feel it's necessary to adopt such a condescending and superior tone both in your comment and in your article.

swombat3• 16 years ago

It's hard to tell someone they failed at something basic without coming off as condescending or superior. I'll try to do better next time, though. I don't like posting negative/attacking stuff, generally. :-/

Confusion• 16 years ago

Would you have taken his comments as seriously if he would have adopted a style of communication where he would be more forthcoming? That other style would have included all kinds of 'nice' words, that distract from the message and provide all kinds of room for you to think that 'it wasn't as bad', 'I only need to polish/fix some details'. I think swombat has done you the favor of driving the seriousness of your mistakes home, making sure you don't forego thousands of dollars, just because he was trying to be nice.

In a face to face conversation, you can be nice all you want, because you have both time and non-verbal oppurtunities to make the point. The internet is different, for better and for worse.

jamesaguilar• 16 years ago

Says the guy who makes his argument with the phrase "I fail to understand." Let's face it, it's hard not to be condescending when we know we're right. For one thing, it has a lot of visceral appeal. For another, it's a strong rhetorical position for convincing people not familiar with the subject matter.

schammy• 16 years ago

I'll just chime in here and say "I agree 100%", since all replies to so far (excluding the OP's) are basically calling you a moron. :)

Yes, the test had some serious flaws, but I imagine that removing the free plan was by far the biggest factor in the results - especially because there's no demo and no trial. I can't imagine signing up for a pay-only service if I can't even try out a demo of it before entering in my credit card.

I found value from the post, even though, yes, it is flawed.

matt15• 16 years ago

Hey, I appreciate your feedback and will improve in the areas where this experiment was weak in future tests.

swombat3• 16 years ago

As others have pointed out, my post came across as overly harsh and condescending. To balance this out slightly, I'd like to point out that I found out about your article not via HN, but via my RSS reader, since I follow your blog! Keep up the good posts! (the ones for which I subscribed!)

cubicle67• 16 years ago

look mate, I've never met you and I have no idea what you're really like, but in your comment here and linked post you come across as quite a dick; one of those people who feels the need to publicly humiliate others in order to demonstrate their superiority

andrewljohnson• 16 years ago

You need more data. This is not particularly significant.

aaronblohowiak3• 16 years ago

I think your example timeline doesn't show off a lot of the cool features you have, and that is unfortunate. In your future A/B testing, you may want to explore a timeline that includes pictures... perhaps of something like the Homestead Act or such.

astrange• 16 years ago

Why would I want a web-based timeline? Isn't that just something you put in a PowerPoint slide and then nobody else cares so they don't read it?

I Eliminated the Free Plan from my Web App for a Month: Here’s What Happened.

Related Stories