How to do AB Testing using Google Website Optimizer

http://www.fogcreek.com/FogBugz/blog/post/How-to-do-AB-Testing-using-Google-Website-Optimizer.aspx

by mgorsuch • 16 years ago

33 7 16 years ago

webwright2• 16 years ago

Having just used GWO for a project because it was a requirement (GWO is integrated with Shopify), I can now much more enthusiastically endorse Visual Website Optimizer.

paraschopra• 16 years ago

Thanks, I take that as a compliment :)

webwright2• 16 years ago

You should! VWO rocks pretty hard.

xal• 16 years ago

we integrate with GWO before there was Visual Website Optimizer or Optimizely. Believe me, i'd love to undo that decision.

aymeric• 16 years ago

The video states A/B testing doesn't help you find the global maximum, which I agree with.

So how do you go about finding the global maximum? Using A/B testing you could create some variations that are 'wild' to hope to get lucky but it is not very efficient. What else can you do?

chmike• 16 years ago

This problem is well known in optimization. The A/B testing approch, when use correctly, provides a local optimization method that will bring you closer to the local maxima (or minima by inverting the metric sign). Note however that it is not the most efficient.

To locate the global maxima, if you have no prior knowledge on the underlying function, you have to pick very different starting points and apply the local optimization algorithm. The starting points may be picked at random, or in a grid for complete scanning or preferably using some priori knowledge which we do have on web services efficiency.

michael_dorfman• 16 years ago

I thought that part of the video was bullshit. I thought the same when Atwood wrote his "Groundhog Day" piece, which the video seems to accept at face value.

A/B testing lets you test two competing hypotheses. It does not in any way require that the two hypotheses be very similar to each other. You can test radically different designs against each other just as easily as you can test minor differences.

So, the real question you are asking is: how do you generate new hypotheses? And the answer is: the same way as always. You can see what other are doing. You can think "out of the box." You can use your best judgment, and whatever creativity you have access to.

How to do AB Testing using Google Website Optimizer

Related Stories