Triple Your Google AdWords CTR Overnight by Doing Nothing

http://www.seobook.com/archives/002026.shtml

by danielhodgins • 16 years ago

68 9 16 years ago

This is a very curious phenomenon. Same everything, yet vastly different click through rate. Have any of you tried this?

terrellm• 16 years ago

From 3 years ago but it certainly still holds true. As mentioned in the comments, many PPC'ers use http://www.splittester.com to help determine if there's enough data to determine a winner.

SEOBook recently opened another site focused primarily on PPC at PPCBlog.com and this was a recent discussion in the forums.

TimothyFitz2• 16 years ago

It's worth noting that splittester.com calculates 95% confidence that those two (identical) ads will have different long term response rates.

ellyagg• 16 years ago

Sure because 95% leaves 5% of room to cherry pick aberrations.

reitzensteinm• 16 years ago

Which should be a pretty big wake up call to anyone not doing the math. 1 in 20 events happen all the time, yet naively reading the results makes it look like one is almost certainly better than the other.

klochner• 16 years ago

Pretty kludgey way of adding statistical significance.

I would expect anyone on HN would be capable of running the numbers and generating the actual stats.

If not, it would be a good learning exercise.

10ren• 16 years ago

Kludgey is right; and it won't always be accurate: numbers might happen to agree on a too-small sample; or diverge on a large-enough sample.

But I think it's a very simple, concrete and credible demonstration to users. And if they observe such an expt over time, over a few trials, they'll quickly develop an intuition for when the sample size is large enough. And this is more valuable than an abstract calculation. (Of course, ideal to have both.)

Unfortunately, the empirical approach doesn't take into account the effect of how near the probability is to 50% vs the extremes 0% or 100%; nor of the variance of the population. However, in practice, these are probably similar enough over all ads and populations for the effect to be negligible - especially if you give yourself a margin of safety.

3pt14159• 16 years ago

I hacked together a gtest calculator in ruby if anyone is interested. I keep meaning to clean it up, but I haven't had time.

http://github.com/zachaysan/Split-Test-with-the-G-Test

danielhodgins• 16 years ago

Can't hurt to try this approach, as it takes approximately 10 seconds to set it up. Will give it a shot, and report back if the results are interesting...

Triple Your Google AdWords CTR Overnight by Doing Nothing

Related Stories