"Mad Libs" Style Form Increases Conversion 25-40%

http://www.lukew.com/ff/entry.asp?1007

by malyk • 16 years ago

270 50 16 years ago

mmelin• 16 years ago

Very interesting. I would think that this could become frustrating to use with too many fields to fill out, but it definitely looks more inviting than a standard form.

Now I'm just waiting for patio11 to apply his godlike A/B testing powers on this and show us the results :-)

patio112• 16 years ago

I don't use lead generation forms so I don't see the big win here.

P.S. If you're using Rails, download A/Bingo. If you're not, email me and I will write you design documents sufficient to create an A/B testing framework in your language of choice.

It should not take godhood to do A/B tests. I want A/B tests to be like for loops: not only will you know them and use them automatically every single time, you will laugh with your buddies about that one time somebody applied to your company without ever having written an A/B test before.

kalid• 16 years ago

I would love to see more client-side javascript A/B frameworks like genetify (easy to embed into blogs, existing apps, etc).

Unfortunately that project doesn't look that active, but it's the best out there now.

diN0bot• 16 years ago

is there an A/BIngo equivalent for django/python?

r00k• 16 years ago

I'm not an A/B expert by any means, but it seems like there were far too many changes made to this form to give credit to just the 'mad lib' aspect. I've seen Patrick (patio11) mention that he's often surprised by how the tiniest change can result in a huge conversion difference. The second form is VERY different than the first.

I think a more accurate headline is "Redesigned form increases conversion 25-40% (who knows why?)"

btucker• 16 years ago

I came here to say this, you beat me. Also, it's important to note the context of this "conversion". It's sending a message to someone you don't know. No one likes to send cold emails, so pre-writing the message takes a lot of the burden off. I'd wager just having the "Comment" textarea pre-filled with a message would have similar impact.

corruption2• 16 years ago

I read the following and my jaw dropped:

"While it's possible these adjustments also contributed to the increase, it's unlikely they were solely responsible for it"

What was he smoking when he wrote that? There's absolutely no justification for that statement that I can see.

lambda• 16 years ago

Yeah, there are a lot of differences here that could increase conversion. The second form actually includes some information about what this message is about, while the first form is completely bare and doesn't clue users in to what they are inquiring about. The first form has a wall of 9 input fields (with no indication of what's required or optional), while the second has 6 with a relatively discreet link for adding an optional seventh. The second has a lot of friendlier language overall, and has the less relevant links at the bottom separated out from the form better, reducing the amount of information that you need to pay attention to when submitting the form.

I think the mad-lib form technique is interesting, and I'd love to see some good A/B testing of just that one change, without all the rest. You could also test reducing the number of input fields, which is also likely important, clearly marking what's optional and what's not, and hiding optional fields behind a link instead of presenting them all at once.

webwright• 16 years ago

All data like this is context/market/user specific.

I don't think the headline or article is wrong, though I suppose he could say, "your mileage may vary".

Regardless, I think it's compelling enough to try it out on a few forms. I'll try it on our site and crow if there's a meaningful change in either direction.

patio112• 16 years ago

They could be more certain with a multivariate test. I count 2^7 = 64 possible variations (counting the madlib/form as just one point of difference), which would take an awful lot of traffic to discriminate between. So I sort of kind of understand the desire to wrap it up in one simple A/B test.

But you're right, of course, there is no reason to assume that the change was from the obvious of the 7 changes instead of from all (or one!) of the other six changes. Did anybody notice that the button call to action changed? I've seen A/B tests where that did 20% by itself. Did anybody notice that the header call to action changed? I've seen A/B tests where that did 20% by itself. Did anybody notice that they added the words "Thank you."? I've seen A/B tests where... actually, I haven't for "Thank you." (if you've got one, please share) But you get the general idea.

eru• 16 years ago

> I count 2^7 = 64 possible variations (counting the madlib/form as just one point of difference), which would take an awful lot of traffic to discriminate between.

A fractional factorial design (http://en.wikipedia.org/wiki/Fractional_factorial_design) of the experiments and other statistic techniques (http://en.wikipedia.org/wiki/Bayesian_experimental_design) may decrease the amount of traffic needed.

warfangle2• 16 years ago

This is one of "those things" that has me slapping my forehead saying, "Why didn't I think of that?"

Great idea, and great to find it perform better. It probably helps alleviate the confusion some people feel inherent in dealing with their computer - the same kind of people who have no idea how to respond to a dialog box with yes/no|accept/decline, but can answer the same question promptly when posed verbally.

grayrest• 16 years ago

The first place I saw this style of interaction is in the novella-weight Magic Ink [1] essay.

[1] http://worrydream.com/MagicInk/

warfangle2• 16 years ago

Wow. This is amazing. Thanks so much for the link - bookmarking for later reading.

ErrantX• 16 years ago

When the original was posted here on HN last year I seem to remeber it was quite widely panned as "too different" (a reasonable concern).

It's nice to see an actual implementation - and that it's not a conversion killer or anything.

But it would be nice to see a clearer example of it being the only change to a form.

ThomPete2• 16 years ago

Also as I have already mentioned. It would also be nice to understand whether the Address is optional in both of the examples.

Right now it looks as if it's only optional in the "after" form. And then it's quite obvious why it converts better.

anigbrowl• 16 years ago

Another thing I like about it that he doesn't mention is the button to 'join' rather than 'submit', with its connotations of soulless bureaucracies.

maxklein• 16 years ago

You know before everybody runs off and starts using this, can we have some independent tests? Remember that "Follow me on twitter here" thing, I don't think it was ever proven that it actually worked apart from the original article, but suddenly, everyone all over the web was using it.

I'd like to see some actual independent, not-the-same-dude tests before I switch to this style.

paraschopra• 16 years ago

Yup, this is exactly why I wrote: You should NOT follow me on Twitter http://visualwebsiteoptimizer.com/split-testing-blog/you-sho...

A/B Testing case studies are good for two reasons:

a) Help create curiosity on what the heck this A/B thing is (good for my product) b) Gives you ideas to test on your website

But it is very dangerous to implement results without testing for your case. Very, very dangerous.

tigerthink3• 16 years ago

Dangerous? How about a best practice? If you don't have time or inclination to test, the first strategy you should use is one that worked for someone else.

dcurtis• 16 years ago

Or "See plans & pricing" and "free 30 day trial on all accounts."

People like following blindly.

Silhouette• 16 years ago

> Remember that "Follow me on twitter here" thing, I don't think it was ever proven that it actually worked apart from the original article, but suddenly, everyone all over the web was using it.

I wish they'd stop writing "You should follow me on Twitter" as well. That's just about the most irritating, patronising thing I've seen in weeks. I am quite capable of choosing how to spend my time, and following the orders of someone whose web site I have just arrived at (probably looking for some actual content, which has been shoved further down the page by some annoyingly over-sized blue bird logo) is not high on my list of priorities. On the contrary, you could just relabel your entire page "Back" and make it clickable to simulate the effect it has on me.

Ah, I feel so much better now. :-)

ThomPete2• 16 years ago

In the "before" form address doesn't seem optional in the "after" form address is "optional".

It doesn't really say whether both forms address fields where optional.

The golden rule is the less you ask about the higher conversion. So if one of the changes they did where to make filling out address optional then it's obvious why it converts better.

If both are optional then I would say the "before" version seems almost to be made bad on purpose.

shaunxcode• 16 years ago

That is funny because the other night I started toying with http://www.promisegraph.com - obviously it doesn't work yet but I really liked the idea of the form being in a sentence so I fleshed that in to see if it was even viable. Backend should be coming along this weekend.

nazgulnarsil• 16 years ago

post hoc ergo propter hoc. just because you see A and B together doesn't mean A caused B.

http://en.wikipedia.org/wiki/Post_hoc_ergo_propter_hoc

hartror• 16 years ago

Like the rest of the commenters I'd would like to see some more fine tuned testing as to why, added it to the backlog for attempting on our signup page so maybe I will get to post some interesting results in a couple of weeks.

hockeybias• 16 years ago

I think teh 'mad libs's tyle helped (a guess), but I agree that the two forms are different in other ways.

neilk• 16 years ago

Interesting and obvious in retrospect. The direct mail industry has been using these techniques for years, with lots of research backing them up.

Full disclosure: in my student days I used to work for a direct mail company, not as a programmer but as a general layout and graphics guy. They weren't mail spammers per se, they specialized in hospital funding drives, or did mail followups on people who were on prescription drugs, to remind them to keep renewing.

Anyway these letters were tested just as carefully as we do today (only an A/B test involved actually printing and mailing stuff). The "Yes! I want to know more about $FOO" format may seem cheesy but it was a proven winner.

You would think people might get tired of it, but they believed that their targets were instead "trained" to respond to these sorts of appeals. That's literally what I was told, when I suggested they try something different.

So, maybe we should be paying close attention to those crappy cards that fall out of magazines?

sunkencity2• 16 years ago

it may look like shit but the direct marketing peoplr really know and use every dirty trick in the book. when it comes to conversion rates they are the best. dealing with a different target group than the average consumer might require different visual style and somewhat ligter touch though

bodhi• 16 years ago

Not to pick on you personally, but people keep mentioning how good direct marketers are. Does anyone have any (scientific-ish) references for this? I'd like to read more about it.

jd• 16 years ago

We've been using a similar approach for Thymer since the beginning and we've always had a really good conversion rate (conversion between visitors and signups):

http://thymer.com/try/

Using a little creativity in the design, even for forms that users fill in only once has been a great way for us to set ourselves apart.

ABrandt• 16 years ago

I knew I had seen this style on an app somewhere around here. Great work, I personally find these forms to be very intuitive. The only suggestion I have for Thymer is for the "re-type password" field. Your prompt there could certainly confuse some users.

juancorbett2• 16 years ago

What he said. Also have you considered that people can type their password and even if they get it wrong it doesn't matter as their browser/password manager can still remember it. And even then they can always just reset it if they got it wrong.

acangiano• 16 years ago

I'm testing this out (http://thinkcode.tv/english). I know my current conversion rate, so I'll be able to see if there is a difference (I'll do the math with next week's data, or else random subscriptions from this group will taint it). :)

juancorbett2• 16 years ago

Your form lacks affordance - it's not clear, despite the subscribe header, that one should complete it by tabbing/clicking and then entering data.

On the OP example at Huffduffer one clicks a "sign up" button (which has the mouseover feedback you lack incidentally) to get to a page which is for sign up. Hence the affordance is not so important as they're already in the "I need to enter data"-pipeline.

Also FWIW I think messing with default button formats should be avoided unless you're going to make sure that it's clearly a button (affordance again).

tigerthink3• 16 years ago

So what are your results like?

hariis• 16 years ago

Help me understand here,

These kinds of form improvements should be done for trying out services that the user already is familiar with, like buying a car, getting a insurance quote, and where there are multiple choices of service providers, right?

Because, on a sign up form for a new web service, I really don't want users to be tricked into signing up. I only want those users who understand what my service offers (through the intro blurb, videos etc) to sign up and if you have decided to try based on the merit, why would a form design matter? If I lost a sign up due to a form design, my service is probably weak, no?

jasonkester• 16 years ago

Ah, but people really hate filling in forms. So much so that if you stick a complicated form in front of something they want to do, many of them will decide they don't want to do that thing so much anymore.

Even something as simple as asking for an email address in addition to username/password will lose you a percentage of signups. (As a real-world example, we tried removing all the fields from the signup page on Twiddla and saw a 1000% increase in users trying us out).

This form design seems to be a way of convincing people that they're not looking at a form, and therefore tricking them into filling it in to go do the thing they wanted to do in the first place.

rwhitman• 16 years ago

A lot of folks commenting sound pretty skeptical - with good reason, of course - but I can't help look at that form and catch myself thinking 'gee, that looks fun to fill out'. I think its really clever.

I'd also suspect seeing your full name in the paragraph would be more reason to use accurate information. Not to mention maybe there could be spam and security benefits in there too

Really like to hear if anyone does some tests against this

aantix• 16 years ago

I went with a similar strategy with a site I put together, JimAsks.me ( http://www.jimasks.me ). I went as far as to create the account for the user with pseudo data, and then giving them the chance to go back and update the account data if they wish. If anyone answered a question, regardless of whether they had created an account or not, I wanted to be able to display that response and by tying it to a persona, I was able to do that. This signup process was presented in a paragraph form similar to the site discussed here (just answer a question you'll get a chance to "change your account" where you'll see my form).

I didn't see any uptick in those who signed up. But the generation of anonymous accounts on behalf of the user certainly did increase the interaction with my site.

tigerthink3• 16 years ago

Suggestions for improving your form:

* "I'm given you" -> "I've given you", "I'm given", "I'm giving you", or "I'm givin' you"

* It's not clear what the field after "from" is supposed to be

* "your email is [ ] with a password of [ ]" -- what do you want my email password for? Try "your email is [ ] and your password is [ ]".

I've been reading Don't Make Me Think and Rocket Science Made Easy, both by Steve Krug. Good stuff if you want to catch issues like these. Just a small amount of usability testing ("a morning a month") can bring significant benefits.

phr• 16 years ago

The first form screams "Add me to your prospects database, so your automation can start your sales people hounding me". The second form softens that impression a bit. I suspect it might work better on me.

vdm• 16 years ago

I think a reason the second form may work better is that explains implicitly how the information you provide will be used.

979s• 16 years ago

I see this as an incredible leap in branding in the most unexpected places (forms) for those sites that want to go above and beyond in appearing personal, innovative and positive. As an A/B test (as has already been pointed out) it proves nothing but deserves further insight.

lunchbox• 16 years ago

There is a significant possibility that the improved results of this format are due to the fun and novelty of filling out such a form. If everyone starts copying it, its novelty may wear off and it may lose its effectiveness.

zach• 16 years ago

This could be a great way to ask users for their feelings when an error occurs, like the Wufoos suggest -- "right now I'm feeling (adjective) about it":

http://particletree.com/features/on-asking-users-for-their-f...

"Mad Libs" Style Form Increases Conversion 25-40%

Related Stories