Bounding Viral Impact in Experiments

http://juneandrews.com/2015/02/27/bounding-viral-impact-in-experiments/

by astuteajax • 11 years ago

9 3 11 years ago

Zephyr314• 11 years ago

Great post. Quick question: Is there a way to artificially bound virality in an experiment so that you don't need to partition the network without destroying the signal?

astuteajax2• 11 years ago

Ah! In some situations it is possible at the cost of a degraded user experience.

Let's say we have an experiment that encourages sending likes. There are two users here, the user sending the like and the user receiving the like. A traditional a/b test flips a coin. If the coin turns up heads, then the user receives the treatment. To control viral spread we want to flip a coin on the interaction of the like. To implement with a traditional experiment framework flip two coins, one for the user who could send the like and one for the user who could receive the like. If both coins turn up heads, then the interaction of the like receives the treatment.

A benefit is it allows us to test 2 hypothesis:

* Sending more likes is good for users

* Receiving more likes is good for users

However, the cost is an inconsistent user experience. If each coin is a 50-50 coin, then for a user sending the likes a random half of their network will be featured. Similarly for the user receiving likes, they may not understand why the sudden upturn in likes from some people when they may feel closer to others in their network.

astuteajax2• 11 years ago

Good Morning! I'm the author of the post and tried to highlight the salient points. Please feel free to ask questions for a more indepth explanation of application, derivation, etc!

I will be actively checking and answering questions today.

Bounding Viral Impact in Experiments

Related Stories