A/B Testing to Increase User Engagement

http://blog.mixpanel.com/ab-testing-to-increase-user-engagement

by trefn • 17 years ago

47 5 17 years ago

blog.mixpanel.com

btilly• 17 years ago

The post has good questions for people to ask, but not specific advice about how to analyze results and decide when you have a conclusion. So I added a link to my tutorial in the comments section. ;-)

mey• 17 years ago

How do you stub out a community?

If the community is engaging, people will be more likely to comment (like HN), but if you are a site, you really don't have two separate communities to swap in/out in the user's mind. Additionally, even if you did optimize your site design/functionality to increase comments, is that really desirable without an engaging community in the first place? Would everything end up being nothing that adds to the conversation? (see Youtube)

richcollins2• 17 years ago

You guys suggest using Mixpanel to measure user engagement via split tests, but Mixpanel doesn't have a good way to split test!

I've resorted to hacking something together using properties. I use the property name to represent a segment/test name and the values to represent the splits. This leads to having a ton of properties, which the UI isn't designed for.

ex:

  {"event":"signup","properties":{"techcrunchreaders/signupbutton":"blue"}}
  {"event":"signup","properties":{"techcrunchreaders/signupbutton":"red"}}

trefn• 17 years ago

An A/B testing tool is on the roadmap, but we just haven't had time yet.

The best way to do tests right now is by utilizing funnel properties. You can test the whole length of your signup funnel and just add a 'button-color' property.

This lets you see the difference in conversion rates between your two treatments pretty easily.

richcollins2• 17 years ago

I would also like to do this for one off events. The important thing for us is the ability to view the event metric across property values (intersecting segments):

{"google/organic":"widgets"}

{"google/organic":"sprockets"}

A/B Testing to Increase User Engagement

Related Stories