Stop guessing. Use A/B testing to determine ideal price for your product

http://visualwebsiteoptimizer.com/split-testing-blog/ab-testing-price-testing/

by paraschopra • 15 years ago

59 59 15 years ago

lachyg• 15 years ago

Good post, I think more people should be testing different values (I remember reading something where someone kept doubling the price til they reached the tipping point).

Cheers

paraschopra8• 15 years ago

Yes, price testing is very important at initial stages of launch. It becomes progressively harder as your product gets established.

After your product is established, the only way to test price sensitivity is to launch pro version.

ambirex• 15 years ago

A pro version isn't the only way, you could launch:

* light version

* experiment with different sales and promo codes

* offer different rates for paying ahead a certain amount

btilly4• 15 years ago

You're thinking about http://jacquesmattheij.com/Double+your+price+%28and+no%2C+I%... which was discussed in http://news.ycombinator.com/item?id=1639712.

wensing3• 15 years ago

Keep in mind that this is secondary to finding the market that puts the greatest value on your product.

paraschopra8• 15 years ago

Interesting. Can you clarify a bit more? An example would really help. Thanks.

wensing3• 15 years ago

Stormpulse has recently shifted from B2C to B2B. Pricing is so completely different there's no A/B test you could use to determine that.

Say you were selling tickets to hear Lang Lang play the piano. You could try selling those on the corner of 3rd and Main and get $20 for them because no one you talk to (unless you get lucky) has heard of him (and you could waste a lot of time convincing them that he's great). Or you could move your booth to the more fashionable part of town and sell them in 5 minutes for $250 each.

rmc2• 15 years ago

Why is it illegal to offer different prices for the same service (as this article says)?

paraschopra8• 15 years ago

Read this: "PRICE DISCRIMINATION AND THE LAW" http://www.imakenews.com/strategicpricing/e_article000649775...

jat850• 15 years ago

The article you linked specifies that the statute in question only applies to goods, not services. Is that applicable here?

gregpilling3• 15 years ago

To quote the article you linked to : "This complex, Depression-era legislation was enacted to protect small businesses by outlawing discriminatory price and promotional allowances obtained by large businesses. As is the case with the other antitrust laws, the Department of Justice, the FTC, and private parties may each bring Robinson-Patman cases, although the enforcement agencies have not focused on this area for some time. Indeed, the Justice Department has criminal powers in this area that have gone unused for many years, while the FTC today brings few significant cases in this area after being particularly active through the 1970s. Private suits on behalf of businesses (consumers have no standing to sue under the statute) account for most of the enforcement activity. Successful plaintiffs are entitled to the same remedies as those available under the antitrust laws (e.g injunctions, treble damages, attorneys’ fees and costs)."

The article lists the very specific reasons a case may be brought, and the various defenses. I do not see how any of this is relevant to your argument that "charging different prices is illegal". I do however agree with your concept that you should try out different prices strategies.

I have been amazed in my own experience when I make a package of my products priced beyond any shade of reason (from my view, as seller) and someone goes ahead and buys it. I had a manufacturing company and the 'big' package was to buy $1200 worth of goods and it was sometimes a tough sale. We tried a new 'big' package that was priced a little over $10,000 and it was an easier sale (to bigger customers). Lesson learned - don't ever think you know what the limits are until you test them. You might think that nobody would ever pay $100 per month for your widget, when some division of a Fortune 500 company would happily pay you a $1000 per month. Try it, you may be surprised.

btilly4• 15 years ago

It refers to the Robinson-Patman act, about which you can find more information at http://en.wikipedia.org/wiki/Robinson%E2%80%93Patman_Act including a link to the relevant statute at http://www.law.cornell.edu/uscode/15/13.html.

My non-lawyerly reading says that you only need to worry if you're shipping physical goods for resale. And even then you have room to be OK as long as the price differences aren't creating a monopoly, and aren't enough to economically injure the people who receive a higher price.

mckoss• 15 years ago

Note that selling a product below its cost is required to break the law (aka "dumping"). I would assume this means below its "marginal" cost; which, for digital goods, is nearly zero.

rmc2• 15 years ago

Interesting. That applies to USA. Is there any EU law on the matter?

gregpilling3• 15 years ago

It is not illegal to offer different prices. Case in point - a lowly 12 oz of Budweiser. If I buy it in a keg it costs me around 50 cents (USD)each. In the bottle it would be closer to $1. At a local bar that has good beer prices and good food it is $3.00 for draft normally, except on Thursdays it is $2 (Bisonwitches, Tucson AZ). If I go to a fancy restaurant it will likely be $4 or even as high as $6.

In a free market you are free to charge what you want. Consumers are free not to purchase your item. Rules do vary around the world, so check with your local authorities. I am looking at this from a North American perspective.

podperson2• 15 years ago

Ah so it's OK to charge people you don't like (wrong skin color say) more because it's a free market?

There's a difference between giving bulk discounts and selling the same product for different prices to different customers.

ekanes3• 15 years ago

I think you're wrong. 12 oz of bud in a can is a different product than 12 oz in a bottle. 12 oz by the can is different than by the case. But if you line it all up and make everything else equal, and have a line of customers in front of your till, it's illegal to charge different prices to those customers in line.

spot• 15 years ago

http://definitions.uslegal.com/p/price-discrimination/

this doesn't address this case directly, but it seems to say it's only illegal if done to harm your competitors.

acangiano• 15 years ago

Pro tip: Have at least three pricing levels with directly comparable features. It will increase the number of sales for the intermediary level.

jacquesm3• 15 years ago

I've heard that one before and I've seen sites that do it but I have yet to find a place where there is actual evidence that it works, do you have a reference for that?

We're debating trying this and I'd like a bit more evidence that it really works and why it does.

tomkinstinch• 15 years ago

Here are a few citations, not all directly relevant, from Dan Ariely's _Predictably Irrational_ (itself a good read for any startup founder):

___________________________________

1. Amos Tversky, "Features of Similarity," Psychological Review, Vol. 84 (1977).

2. Amos Tversky and Daniel Kahneman, "The Framing of Decisions and the Psychology of Choice," Science (1981).

3. Joel Huber, John Payne, and Chris Puto, "Adding Asymetrically Dominated Alternatives: Violations of Regularity and the Similarity Hypothesis," Journal of Consumer Research (1982).

4. Itamar Simonson, "Choice Based on Reasons: The Case of Attraction and Compromise Effects," Journal of Consumer Research (1993).

5. Amos Tversky and Itamar Simonson, "Context-Dependent Preferences," Management Science (1993).

6. Dan Ariely and Tom Wallsten, "Seeking Subjective Dominance in Multidimensional Space: An Explanation of the Asymmetric Dominance Effect," Organizational Behavior and Human Decision Processes (1995).

7. Constantine Sedikides, Dan Ariely, and Nils Olsen, "Contextual and Procedural Determinants of Partner Selection: On Asymmetric Dominance and Prominence," Social Cognition (1999).

paraschopra8• 15 years ago

In 2000, Amazon got caught doing price testing and the news was covered on all major blogs. Here is a link: http://news.cnet.com/2100-1017-245631.html

mattmanser• 15 years ago

What's your source on a/b testing price on identical products being illegal?

I've never heard that and none of the links provided in the comments so far justify your claim. It is a pretty extreme assertion and you provided no links justifying it either.

I can see this rapidly turning into an urban myth, whispered wild eyed at 3am over ramen noodles as founders try and figure out a price for their product.

davidw2• 15 years ago

I don't know if the other comment was edited, but he doesn't say anything about 'illegal'. People certainly found it distasteful though.

paraschopra8• 15 years ago

The concept is called "price discrimination". There is a great discussion going in this thread and I may be wrong at this. Even if it may not be illegal, it would leave a bad taste in your potential customers' mouth especially since there are better alternatives.

ekanes3• 15 years ago

I don't believe it's legal to charge different prices to different people for exactly the same product. Changing the price (for everyone) is fine (you could even change the price 5 times a day), or adding a back scratcher and bumping the price by $500 is fine, but doing an AB test where you flip the price back and forth on the same thing isn't legal as far as I know. IANAL.

podperson2• 15 years ago

The problem with doing this is that if your customers find out they're likely to be pissed off. If this doesn't matter to you then feel free.

btilly4• 15 years ago

A major complication that you do not address is how to do confidence testing on the results of your pricing difference. There is an important difference between "A did better than B but who knows why" and "We're confident that there is a real difference." Figuring that out is not easy.

paraschopra8• 15 years ago

Yes, that's right I should have touched that point. I think the revenue for each variation can be approximated as normal curve and then a hypothesis test can be done to see if difference in mean is statistically significant.

btilly4• 15 years ago

That approach absolutely works. But fairly few people have enough of a math background to figure out how to do it.

There are also complications around noise from large outliers. Though this matters less with a fixed price service.

michaelleland• 15 years ago

"Pricing for physical goods is simple. For example, if you are manufacturing staplers, all you need to do is to calculate cost of production and distribution, slam 20% margin on it and there you have the price you can sell your shiny stapler machines for."

Not true. In fact, a lot of Maker businesses get hung up here. The pricing of physical goods depends on the market every bit as much as the pricing of software or web services. In fact, large companies rely on that all the time--case in point is P&Gs Swiffer Sweeper, that they charge much more than the manufacturing cost for the sweeper and for the refills.

Many small business owners I've run across actually feel guilty if they charge "too much" (say 100% markup) for their physical goods they are selling, even though their competitors sell at that price. The price elasticity of demand applies just as much for physical goods as it does for non-physical.

roel_v3• 15 years ago

To an extent I agree with the sentiment of your post, but the practicalities are different imo, and "The pricing of physical goods depends on the market every bit as much as the pricing of software or web services." I don't agree with at all. Many non-software products and services (staples, pies, chimney sweeping, plumbing, ...) are much more commodities than software. Efficient markets, lemon problem etc. drive down the markup to a low (single to low-double digit) percentage. Since cost is also much less variable (costs in terms of resources, labor, capital investments etc. for a pie from one baker is going to be very similar to those of another down the street) the end-price is going to be not very flexible; and any eventual elasticity will just drive down consumption rather than price. The degree of availability of substitution options is a major influence. That's why even major supermarket chains operate with low single-digit profit margins, even on a product (food) where there is obviously a constant and high demand.

Compare that to software where no two offerings are direct substitutions (the degree varies, of course) and where one company can upset a market with new technology very easily without his competitors necessarily being able to duplicate (for skills reasons, legacy investments, ...). An incumbent can enter the market with fixed costs that are 75 or 50 or 25 percent of the established players. Such an incumbent can do two things: go deep on price to get market share, or get fat off 500% profit margins. I don't know of many such situations in physical goods businesses.

(of course the same could happen with physical goods, it's just much more common in technology)

ZoFreX• 15 years ago

This article was overall a very shallow economic analysis, they should have gone into marginal costs and price differentiation for this problem. Maximising income from customers by charging more to those willing to pay more is a very well studied problem, and solutions to it are observable in every shop on the high street.

rcavezza• 15 years ago

I don't like the first paragraph.

"For example, if you are manufacturing staplers, all you need to do is to calculate cost of production and distribution, slam 20% margin on it and there you have the price you can sell your shiny stapler machines for."

That's called Cost Based Pricing and is one of many options. That's a decision that was made. You can argue that's the wrong way to price. If you make something in the US and charge 20% on top of costs and have a competitor making Staplers in China, you're out of business.

Boom. Roasted.

phylofx• 15 years ago

Well, if one has to offer "completely" different products at different prices for testing purposes (say product "x", "y", "z") like the author suggests, how can you determine the ideal price for product "x" in the end?

stcredzero• 15 years ago

Any best practices for A/B testing on the iOS App Store?

RobertKohr• 15 years ago

I have done this with an iphone app. The ideal case for testing:

-Your app has little publicity and is generally found through keyword searching. This means that you have few repeat visitors to the app page

-Your daily sales data has little variation. You typically see this 2 weeks after release when it is no longer in the new apps list. Avoid holidays and other sources of variation for your test.

-Schedule pricing into the future on one week increments. Try not to have pricing go generally up or down, perhaps by using some randomization or shuffling of possible prices.

-Use the revenue generated during each of the weeks coupled with the pricing to identify your ideal price.

Using poor scientific measurements (I crossed holidays, and did a declining price from a high point) I came up with $2.99 as an ideal price for my app.

Keep in mind that your sales rank influences your revenue, and if you think your app can break the top 100 apps list, then price it at $.99, because once you get into that list any losses prior due to underpricing an app are nullified.

Too bad the app store doesn't primarily sort by revenue (aka value to the consumer) and doesn't provide a continuous scroll past the top 100 which would make such devaluations unnecessary.

rabidsnail• 15 years ago

Has anybody tried using hill climbing to arrive at an optimal price?

eg:

1. Pick a pivot price

2. Make N buckets with prices at fixed intervals from the pivot

3. Find the bucket with the most profit, and use its price as the new pivot

4. Repeat

khafra• 15 years ago

Fascinating concept. Makes me want to start selling something in large volumes, just so I could apply other optimization techniques like simulated annealing or beam search to price points and other scalar variables.

bnoland2• 15 years ago

If 99% of your value is in the base product and 1% is in the advanced features, most people will chooses the cheapest plan available.

My customers don't care about ssl or extra storage so they'll always choose the cheaper plan. But if the basic plan was +$10 what percentage would still buy it?

jules• 15 years ago

He is not saying you should offer a second basic plan next to the primary basic plan. He is saying that you should AB-test two different basic plans, only one of them is visible for any given customer.

bnoland2• 15 years ago

What if you do display different prices, but at the end of checkout you surprise them and charge everyone the lowest price in your range.

jeremydavid• 15 years ago

That is a really interesting idea.

The only issue I would see with this is that having the sudden "We lowered the price!" message would influence conversion rates at the final step.

You would have to measure and compare the number of "add to baskets" and "proceed to check outs" and not the revenue / conversions.

Stop guessing. Use A/B testing to determine ideal price for your product

Related Stories