Denial, anger, bargaining, depression, & acceptance: The A/B testing lifecycle.

http://benplesser.com/2012/09/20/denial-anger-bargaining-depression-and-acceptance-the-ab-testing-lifecycle/

by bjpless • 14 years ago

24 11 14 years ago

benplesser.com

bjpless4• 14 years ago

What is the most frustrating A/B test that you've run?

jack7890• 14 years ago

Are you A/B testing different Kevin Costner photos right now? Clever. I like it.

forgot_password• 14 years ago

Off-topic but what do you guys use to run A/B tests?

bjpless4• 14 years ago

We're currently using Google Content Experiments. The Pros are that it's incredibly easy to get a test started thanks to the presence of Google Analytics scripts in the rest of the website. The major Con is the lack of true multivariate testing.

We experimented with Optimizely but had a bad experience. It seemed as though their javascript was synchronous and blocking. We may have been able to customize that but did not dig deeper.

bjpless4• 14 years ago

For our logged in user A/B testing (often relating to email), we've built an in-house solution that uses a MOD based system to track group performance (as opposed to storing each group member in our databases).

Pretty clever implementation by the data scientist on our team.

karolisd• 14 years ago

We've looked at Optimizely and Google, but we prefer VWO.

btilly• 14 years ago

Getting emotionally invested in your tests is a good recipe for an emotional rollercoaster. Don't do that.

Instead try a bunch of things, and see what sticks. Be curious, not invested.

suneel0101• 14 years ago

That's a great point, echoing the part of this post about not being results-oriented. Process-driven is the way to go!

thenomad• 14 years ago

This is a great piece. I'm doing a lot of testing for advertising right now, and the emotional rollercoaster - no matter how sanely you attempt to approach it - can be pretty intense. It's good to see someone writing about that side of the process!

bjpless4• 14 years ago

thanks thenomad. There's a funny disconnect between the rational, detached self and the insane subconscious self present in most people.

Denial, anger, bargaining, depression, & acceptance: The A/B testing lifecycle.

Related Stories