EA “Gives Away” 1000s Of Free Games Due To No Server-Side Validation

http://minimaxir.com/2012/10/client-side-validation-is-hard-mode/

by minimaxir • 14 years ago

135 103 14 years ago

minimaxir8• 14 years ago

(I apologize in advance for linking to my own blog, but I had not seen this issue reported on any tech blog nor appeared in the "new" queue on HN, so I wrote a quick post this morning on the issue, as it's technically interesting.)

esrauch• 14 years ago

Sorry to comment off of the content of the post, but when I load the page the Like button flickers shown/not shown a bunch of times causing annoying visual jitter. I'm on Chrome 22 if that is relevant.

minimaxir8• 14 years ago

From what I can tell, that's on Facebook's end, as all the Facebook plugins seems to flicker occasionally as well. I'll look into it, although there's nothing unusual with my front-end code that should be causing unusual behavior...maybe CloudFlare caching issues?

Thanks for letting me know btw. :)

minimaxir8• 14 years ago

I figured out the error (I was wrong, it was indeed my code) and fixed it. No more flickering. :)

gabemart• 14 years ago

Is there a problem with linking to your own blog?

mparlane• 14 years ago

Self promotion [of your own blog/site] is frowned upon I believe. We have SHOW HN: for when you want to show something off that you have done.

I think it's mainly to stop reporters at Gawker from linking to all their articles though ;)

rcfox• 14 years ago

That's okay. You did a better job than what any of the popular tech blogs would have done.

Negitivefrags3• 14 years ago

> The proper way to implement a promo code is logical: when the user attempts to apply a promo code, the server checks, has the user used this promo code before? by querying SELECT FROM transactions WHERE user_id = current_user AND promo_code = “OS3874XVC”. If the result set is empty, then the user hasn’t used the code, and everything’s good to go.

Be very careful here, it's not as simple as it looks. If you don't do the correct locking it is very easy to have timing attacks which allow keys to be used twice.

We learned this the hard way after posting a one use key for our game on 4chan. A thousand people rushed to try the key and around 100 people managed to get in using it.

ams6110• 14 years ago

It really isn't as easy as it sounds, is it? A better approach might be to UPDATE promo_codes SET used = 1 WHERE ... AND used = 0, and then check that one update occurred. Or if your db supports it, SELECT ... FOR UPDATE OF ... and then update if the code is not yet used. You want the check and the update to occur atomically. And this varies depending on databases, for example in Oracle even if you explicitly begin a transaction, a select does not lock the row (unless you specify FOR UPDATE OF) so if you do the check and the update in separate statements, you still have a race condition).

nvmc2• 14 years ago

Just out of interest, what is your game?

numair• 14 years ago

... That's not a timing attack. A timing attack involves measuring the time taken to return an invalid response to a password attempt and, given enough tries, figure out the password based on this. It usually takes advantage of the way that string-matching code is written in libraries. Preventing this is not nearly as simple as you may think; for example, recent research shows that over a long enough period of time and enough attempts, it is possible for an attacker to factor network latency into their timing analysis.

What you're describing is not the result of an "attack," but rather the result of code that wasn't designed to deal with database locking. You weren't hacked, nobody attacked you; you just didn't design your system to deal with tons of people trying to write/read the same thing at the same time. Again, it's a tricky problem, so it's understandable (and a lot of people would consider dealing with such issues to be "premature optimization").

I will also note that the OP needs to stick to his MySQL Cookbook rather than commenting on coding practices for large-scale, heavy-usage web applications. His code suggestion is terribly naive, arrogant, and embarrassing. Yes, EA made a mistake; no, you have no clue what the hell you are talking about.

NoPiece• 14 years ago

I don't think you have characterized the problem accurately (or at least completely). The coupon allowed you to add dozens of games to your cart, then applied the $20 off to each individual item instead of just one, or against the total. It was a logic problem in applying the coupon on the server side.

Evbn3• 14 years ago

Please change your CSS so it doesn't force Android to show a microscopic font with long lines and disabled zoom/ reflowing.

minimaxir8• 14 years ago

The CSS is based off of Bootstrap, so everything should be showing correctly on mobile and tablets (I don't have an Android device myself so I can't verify that particular use case). I'll look into it; in the meantime, I enabled zoom for mobile devices.

CamperBob2• 14 years ago

in the meantime, I enabled zoom for mobile devices.

Whoever turned it off in the first place was a bad person, and should feel bad for doing so.

ricksta• 14 years ago

I'm not sure if op's solution is optimal. But why not use unique promo code?

dfox• 14 years ago

Unique promo codes have to be generated somehow. Integrating that generation into probably outsourced survey mechanism is likely not exactly trivial and may (if not done right) open ways for anyone to generate any amount of promo codes they want.

This is one kind of problems you get when you start separating different functionality into separate applications that don't known about other parts.

marek12886• 14 years ago

The above mentioned approach is significantly simpler to implement and easier to measure (when it works! ha) than uniquely generated promo codes. So you're also looking at a cost/benefit trade off.

andypants• 14 years ago

Your proposed solution is not that simple. You make it sound like EA simply used the wrong query.

> One, if EA is technically incompetent enough to allow such a severe bug to exist, they won’t have the technical skills to discern who used the promo code more than once.

The bug was not that obvious, and doesn't necessarily imply lack of technical skill. On the other hand, finding out who used a promo code more than once, as long as these things have been logged, should be trivial for any admin.

ColMustard• 14 years ago

That a promo code could be used more than once, or that you could use it in ways it was never meant to? Of course the problem it's not simple, but anyone who thinks it is shouldn't be working on these things. Either someone screwed up building the system, or someone thought they could use it in ways it wasn't intended and screwed up. Sure, the solution might not be as simple as a lack of server-side validation, but either way, the solution is simple, the problem isn't.

catshirt• 14 years ago

what then, in your opinion, is an "obvious" bug? this is really about as obvious as it gets, from a development and qa perspective.

tantalor• 14 years ago

IANAL, but it's possible that these users are now liable to pay for the extra games. They knew that the code was only usable once, yet used it multiple times. EA might ask for the games back or charge the users.

(I'm not taking EA's side, I'm just pointing out some possible consequences.)

spuz• 14 years ago

Can you imagine how much worse EA's image would get if they did that? I'm hoping the PR department will override the legal department in this case.

rabidsnail2• 14 years ago

If they did try to charge for the games they would get enough chargebacks that they would have a hard time processing credit card payments for a while. And with chargeback penalties they might actually lose money.

The only sensible options are a) do nothing or b) revoke games purchased with these codes. If I were them I would do nothing and treat it as an unplanned pricing experiment. Since a lot of these games have an online component (network effects!) the "giveaways" might increase real sales overall.

oakwhiz2• 14 years ago

I doubt there would be chargebacks since a valid credit card was not even asked for during the process of entering the code.

Havoc2• 14 years ago

>EA might ask for the games back

Given the new trend of not actually "owning" the games, I think this might prove difficult from a legal perspective.

xsmasher• 14 years ago

How so? EA can flip a switch to remove the games, and claim there was never a valid license. As long as EA honors the orginal terms of the coupon, I don't see how the "cheaters" were wronged.

ConceptJunkie2• 14 years ago

I would think that the transaction was perfectly legal, and that EA could in no way force people to pay for the games. I wouldn't however, be opposed to the idea that EA could revoke ownership of any subsequent free games obtained once the code had been successfully used, but it would be a PR nightmare if they did.

EA's only reasonable option is to eat the loss and write the cost down as a "lesson learned".

Isn't there precedence to this? For example, cases where Amazon priced something wrong and people bought at that price, even through the the price was almost certainly unintentional. I would think Amazon would have been obligated to honor the bad price until they discovered and changed it just like companies are on the hook for honoring mistakes in ads (although I think they might be covered by disclaimers these days).

romaniv• 14 years ago

I think this is the best possible example of why you should always develop websites using progressive enhancement. First, write the app without using any JavaScript. Test it. Make sure the core logic works as intended. Then add JavaScript to make it faster, pretties, simpler to use. That will most likely prevent fuckups such as this, and it will also result in better structured, easier to reason about architecture.

aero142• 14 years ago

I don't know what this is going to cost EA, but this may end up being a good thing. People hate Origin but more importantly, all their games are already on Steam, so they just continue to buy from Valve. This might get a few people to keep Origin installed on their computer and help the Origin network effect out. Up until now, EAs only strategy has been to make their big name franchises Origin exclusive. This will probably work better and may not end up costing them much.

brainfed2• 14 years ago

They actually pulled most of their top titles off Steam when Origin launched. Mass effect, Dragon Age etc. If you already own them on Steam then they remain in your library but you can't buy them on there anymore.

bluedanieru• 14 years ago

Dragon Age 1 and Mass Effect 1 & 2 are still available on Steam. Interestingly, they pulled Crysis 2 then a few months later added Crysis 2 Maximum Edition.

For Dragon Age 2 and Mass Effect 3, I agree with their reasoning. Steam doesn't allow in-app purchases that circumvent their distribution system, and that's a load of crap.

Tipzntrix• 14 years ago

You ever wonder if this was a strange viral marketing campaign?

SquareWheel2• 14 years ago

This article missed the earlier Reddit thread where a separate exploit was discovered, that the coupon applied to every item in the basket. $20 off of everything. Coupled with a coupon from a different forum you could add tens or maybe hundreds of games to your account for free in one large bundle.

I'm really curious to see EA's response.

fwr• 14 years ago

Do you have a link to the earlier reddit thread?

SquareWheel2• 14 years ago

I do.

http://reddit.com/11ecl5/

The link to the SlickDeals exclusive deals were removed as well for some reason.

http://store.origin.com/store/ea/en_US/DisplayCategoryProduc...

prezjordan• 14 years ago

So because of DRM, they're fully capable of taking those games "back" (removing them from accounts), right? I mean, if VALVe one day decided I no longer can have TF2 they can simply strip it from account, is that how their terms work?

skymt2• 14 years ago

Yes and yes. Valve has been known to remove games from Steam accounts in cases of abuse, e.g. credit-card fraud or stolen license keys.

doesnt_know• 14 years ago

I've never heard of Valve removing single games from an account before. It's common knowledge though that if you ever do a charge back for a purchased game they shut down your account and it becomes irrecoverable.

It's a blanket policy and a reason why I never use Paypal for Steam, use a real or "throw-away" credit card for Steam purchases. Alternatively, you could not buy from Steam at all, I don't anymore but it's too late for me. I already have over 200 games with them and worry about losing access to everything.

cyber• 14 years ago

Ironically, EA will probably try to claim the "success" of the promo by using numbers for all the games claimed as if they were all discrete individuals.

While at the same time trying to claw back the licenses.

tlrobinson• 14 years ago

"pretty embarrassing for an exploit that causes a significant amount of lost revenue"

I doubt much revenue was lost. How many of these people were going to buy the game in the first place? None of them were hot new games, were they?

dagrz• 14 years ago

Is it me or does the author of this article, and the abusers of the exploits he writes about, land on the wrong side of both the law and common morality?

Surely EA being a "terrible company" has nothing to do with whether it is okay to steal their products? Moreover, just because there was a coding error/oversight, again doesn't mean it is okay to steal their products? If you have a complaint about a company or discover an exploit, surely there are other more ethical channels to pursue the matters?

For the record, I dislike some of EA's conduct as much as the next person.

minimaxir8• 14 years ago

I don't approve of abusing the exploit, and I did not abuse the exploit. I put that paragraph in because I thought it was interesting that some people were indeed using that argument as justification for using the promo multiple times.

dkokelley6• 14 years ago

Yes this rubs me the wrong way. I think it's analogous to a retail store (say, an Apple store) leaving their front door unlocked overnight. It's possible to go in and take merchandise. After all, it's not your fault they left their front door open. You could even argue that Apple's business practices are morally questionable, so they deserve to be taken advantage of . (I'm not trying to make any statement about Apple. It's for the analogy.)

You could argue that the situation is different with virtual goods, since they have an incredibly low marginal cost, but I think that the situations are morally analogous. The games aren't supposed to be free.

glesica• 14 years ago

Not really. It's more like Apple issuing you a coupon for a free iPod Nano, but when you go to checkout with the iPod Nano and a Macbook in your cart, the cashier tells you they're both free.

It may still be unethical, my point is just that there are shades of gray here.

eumenides1• 14 years ago

I think the analogy is wrong. EA didn't lose any physical copies.

I think a better analogy would be Chapters/Barnes & Noble (A Book Store) had accidentally put in free to use high quality photocopiers inside their store. The photocopiers were intended to be free to use, but not intended to be used on the books in the store.

Your argument isn't morally analogous because theft implies EA were deprived of something (EA still can sell and play their games), when the issue at hand is the EA botched up controlling access to their product. In your argument, Apple can't sell the stolen merchandise any more.

The more important part for EA would not to be to "punish" or claw back copies. That genie is out of the jar. They should just chalk it up to marketing and move on (fix the technical issue).

milfot3• 14 years ago

I think it is a little more nuanced than simply being 'not okay'. Whether it is 'okay' or 'not okay' is a personal moral decision, which though I might agree with you (for myself), I would not push that morality onto someone else. I would however give a reasoned argument as to why I think it is or is not okay. Ethics are not clear-cut, and ethics do not equal the law.

As dkokelly points out, this cannot be thought of as lost revenue. The vast majority of games taken here would never have been bought otherwise, and it is unlikely that any of these games will be devalued by having a wider audience. So it is spurious to describe this as simply stealing.

EA has a history of disrespecting their customers' privacy, security, payments, and computers, as well as (as the op mentions) questionable ethics. They don't seem to value their customers except as a cash cow to be milked to death. This is a message, an imperfect, impure, but pretty bloody strong retributive message. To draw a long bow, think LA riots.

But still, does that make it okay? I just think each person needs to assess that for themselves. It certainly has a ring of just desserts about it. Speaking for myself, it would depend on my motives. I would put that if the gamers are more interested in getting a positive response from EA than they are in free games, I would suggest they protest by dumping their EA games in an online equivalent of a big bonfire.. 4chan maybe :|

But I would agree that a more ethically clear-cut channel could be actually more effective - but just not because it is more ethically clear-cut. if they really want a positive response from EA, rather than just to protest (or steal), they need to publicly and loudly stop buying EA games.

dkokelley6• 14 years ago

Can this really be considered lost revenue? It's unlikely that all 1,000's of the "free" games were going to be purchased at the current price. You run in to the same questions with piracy. Were the pirates really going to pay if they couldn't pirate?

The lost revenue should really only come from customers who would have paid the asking price but managed to get an illegitimate deal, plus whatever support and overhead costs can be applied to the game downloads.

ntkachov• 14 years ago

Yes because some (most of the ones worth paying for) of the games require some sort of server. So every new player in a game is not free for EA. Giving away a game means they will lose money on this person if they play online.

timmclean• 14 years ago

Although, that'd be an increase in expenses, not lost revenue...

mesozoic2• 14 years ago

Consider however you have to download Origin to get the games and play them. A feat they've been struggling to get gamers to do, so every gamer that got free games is actually worth a lot to them and probably wouldn't have bought any of these games anyway as they're mostly old and lame.

It's pretty much a win for EA which is why I expect they let it continue so long.

Tyrannosaurs• 14 years ago

Someone needs to come up with a term for this, "hypothetical maximum lost revenue" or something.

I don't have an issue with anyone quoting this sort of number (retail cost x lost units) but they need to make clear exactly what it is, which isn't actual revenue or profit.

mesozoic2• 14 years ago

How about "Completely Unrealistic Projection of Revenues Arbitrarily Indexed as Lost" CUPRAIL

elliottcarlson• 14 years ago

"One, if EA is technically incompetent enough to allow such a severe bug to exist, they won’t have the technical skills to discern who used the promo code more than once."

Not necessarily true - we don't have insight in to how their system is setup; and while it may not be tracking redemption of the codes on that level, it could also be attached to any transactional data that they have in place (for reporting purposes etc).

TwoBit• 14 years ago

Also, it's likely the person(s) who set up that promo had little to do with the people who maintain Origin. And may not even be related to those who maintain the server.

chris11• 14 years ago

According to a community manager. EA is honoring all sales made with the coupon. I was kind of expecting EA to ban some people or revoke access to some games. But I guess they just ended the promotion early. Although EA does seem to be getting some complaints from people who filled out the survey and didn't get to use the coupon.

http://forum.ea.com/eaforum/posts/list/60/9040620.page

minimaxir8• 14 years ago

That appears to be the case. I've updated the post accordingly.

xedarius• 14 years ago

I'd say it worked out pretty well, I'd never heard of EA's Origin service before this article.

atas• 14 years ago

This is just another sample case demonstrating how otherwise intelligent and competent people really need to be educated about web development security.

ragsagar• 14 years ago

i can't believe that they don't know what is server side validation.

EA “Gives Away” 1000s Of Free Games Due To No Server-Side Validation

Related Stories