A/Bingo split testing now on App Engine, built for Khan Academy

http://bjk5.com/post/10171483254/a-bingo-split-testing-now-on-app-engine-built-for-khan

by dmnd • 15 years ago

69 8 15 years ago

bjk5.com

patio112• 15 years ago

Hearing about this made my week.

wiradikusuma• 15 years ago

Hearing about this make me sad because my App Engine is Java-based :(

patio112• 15 years ago

Some options:

1) The code is available under MIT and the docs are CCed and specifically designed to make it easy to clone this. At least three people have successfully done so, in a variety of programming languages, web frameworks, and technology stacks. If programming Java is not difficult for you, block off about a week or two and bam problem solved.

2) If you have access to vast amounts of cash but your programmers are just too busy delighting people to write an A/B testing framework, you could hire me to rewrite the whole codebase in Java. This would only cost you low five figures.

3) If you don't want to program and don't have tens of thousands lying around but do have tens lying around, Visual Website Optimizer is a great option for $CHEAP.

4) If you have neither programming skills nor tens of dollars then you may want to reevaluate whether A/B testing is a huge need at this point in your life.

jasonkester• 15 years ago

Here's my C# port:

http://www.fairtutor.com/fairlycertain/

I'd be surprised if you managed to take more than 5 minutes translating the core so that it compiled in Java. From there, it's just a matter of removing the ASP.NET usercontrols from the package and (optionally) wrapping it in your Java framework's equivalent.

petewailes• 15 years ago

Huge congrats on this!

jedc• 15 years ago

Wow... this is awesome! I was literally looking around this week for exactly this kind of framework. Fantastic work, and definitely plan on using it.

Thanks to Ben and Khan Academy!

euroclydon• 15 years ago

Troy Goode wrote a port for ASP.NET MVC, ABsolutelyMaybe. I've had good luck with it. It wasn't too difficult to migrate from the default storage engine, which is serialization.

https://github.com/TroyGoode/ABsoluteMaybe

cavedave• 15 years ago

I do not think you can just stop the test when you find something you think is significant

http://www.evanmiller.org/how-not-to-run-an-ab-test.html 'If instead of deciding ahead of time, “this experiment will collect exactly 1,000 observations,” you say, “we’ll run it until we see a significant difference,” all the reported significance levels become meaningless. '

Reading the stats.py it looks like you might be doing this.

A/Bingo split testing now on App Engine, built for Khan Academy

Related Stories