Show HN: A/Bingo for PHP

http://github.com/waverz/cog/blob/master/cog-guestbook/engine/cog_splittest.php

by sdrinf • 16 years ago

33 12 16 years ago

sdrinf2• 16 years ago

Cog is a little webdev framework of mine, a work-in-progress steal from spending couple of years in different companies.

Since A/B testing is considered pretty fundamental to most of my clients, I thought I'd share with the dear community of HN one possible implementation of it. You can see A/B testing in action on the sample page:

http://github.com/waverz/cog/blob/master/cog-guestbook/pages...

When used together with scriptor, an A/Btest is about 1 line in the design file (db rows, and counters are automagically set up on first page visiting). Stand-alone you might want to integrate it with a templating engine of your own :)

Code is PD, and review is more, than welcome.

Nycto• 16 years ago

> review is more, than welcome.

You opened the door, so here is my feedback:

- You never have to explicitly define any of the super globals as global... you can delete any line that says "global $_SESSION;", for example

- The database interactions look like they are heavily dependent on global state. It is also a tightly coupled dependency

- Use __construct for constructors instead of the class name: http://www.php.net/manual/en/language.oop5.decon.php

- Foreach is better than a for loop for iterating over arrays

- Single character variable names only make code more confusing. You write code once, you read it a dozen times.

I challenge you to rewrite this with only using classes (no global functions) and to write unit tests for your code. Remember, if you have to jump through hoops to test your code, you need to rethink what you're doing.

lmz• 16 years ago

That resolvequery() function in cog_database.php looks interesting. Wonder what happens if you call resolvequery('INSERT INTO ... VALUES (@email, @example)', array('email' => 'test@example.com', 'example' => 123));

patio114• 16 years ago

Thanks -- PHP developers need better alternatives for A/B testing than Google Website Optimizer. Your code is about the minimum viable A/B testing library for a developer: you can use it to start getting your feet wet with A/B testing done the right way.

I think if you start using it in production you will quickly understand that some of the implementation choices you've made are going to cause you pain, but that is less important than doing A/B testing. You can always rewrite the internals later. (I've done it a few times.)

sdrinf2• 16 years ago

Thanks.

Would you mind pointing out implementation choices, that will come back biting us down the road? :)

patio114• 16 years ago

Your mileage may vary, and I don't guarantee I'm reading your code right, but:

1) You will find it useful to be able to coerce A/B test values, particularly in testing and sometimes (for CS purposes) on the live site.

2) I do not see a convenient method of having Ms. Smith get the same website at home and at her workplace. That may or may not be a problem for you.

3) You may find "control" and "experiment" to be limiting once you start doing this frequently -- they aren't nearly as self-documenting six months down the line when you're staring at your web page wondering "What possessed me to make this button teal?!"

4) I'm pretty sure you have a race condition for creation of new tests. (This has bitten A/Bingo a few times.)

aliasaria• 16 years ago

As an alternative, I wrote a port of Vanity for PHP that we use for A/B testing. It uses redis to store data

http://github.com/aliasaria/PHP-redis-A-B-Testing

armandososa• 16 years ago

This is an honest question: is REDIS the best option for storing A/B tests or is just a matter of taste? MongoDB or even MySql could work?

patio114• 16 years ago

I try to keep most of the A/Bingo work in a cache store (Redis, by choice) because sites with traffic two orders of magnitude or so higher than mine would blow up if they used MySQL for it. At something like Facebook game scales I think it would be smart to take A/Bingo and rip the database out entirely.

Writes hurt. A/B testing requires, minimally, O(tests * participants) writes. (And more reads, but the disk will be on fire from writes first.)

However, despite the cost, A/B testing benefits minimally from ACID. If you lose 2% of writes, oh well, it comes out in the wash anyhow. Thus the huge hardware/maintenance budget you'd need for web scale A/B testing on an ACID architecture makes little sense.

Show HN: A/Bingo for PHP

Related Stories