Coding Cloud Heroku May Face Lawsuit Over Bait-and-Switch Claims

http://www.wired.com/wiredenterprise/2013/03/hieroku/

by wwarnerandrew • 13 years ago

178 96 13 years ago

>Heroku declined our request for further comment. “Unfortunately, we can’t publicly comment on legal matters,” said spokeswoman Dana Oshiro in an email. “Heroku is committed to our customers’ success and focused on delivering the best possible product and experience. We’ll continue to update our customers via our blog.”

I don't run a company, and I'm not responsible for maximizing return on my shareholders' investment. So maybe I lack the mental schemas that make these sorts of statements seem reasonable, but they always strike me as short-sighted. Yes, admitting wrongdoing may open you up to legal liability. But it seems to me that, in the long run, it's far better to take your licks and own up to your mistakes than to hide behind the shield of pending litigation. My gut instinct is to offer a sincere mea culpa and provide a transparent roadmap to improvement, and if you lose a court case then so be it. I'd like to think that the good will and respect you build from accountability and honesty will ultimately outweigh any short-term losses from court settlements or judgements.

As I said, I don't live in that world. I would be saddened, though not necessarily surprised, to learn that I was wrong.

biot• 13 years ago

What makes this proposed class action suit possible is that Heroku did admit[0] that their service didn't match up with what their documentation and website claimed, that they messed up, and would be fixing it. And it wasn't something like "our documentation and site could have been misinterpreted", they actually admitted the website and documentation were inaccurate with metrics which were incorrect. It's like a class action lawyer's wet dream of culpability on the part of the defendant.

[0] https://blog.heroku.com/archives/2013/2/16/routing_performan...

kyrra• 13 years ago

Seeing that there is a website/lawfirm[1] threatening legal action, I can bet Heroku/Salesforce lawyers jumped on that ASAP and let all their employees that are allowed to talk publicly not to comment on the issue anymore.

The US legal system is setup that if a company admits fault in something like this, it makes defending yourself much more difficult. Meaning if a lawsuit is brought about, it's not about proving innocence anymore, it's about mitigating the damages you will have to pay. Your lawyer is going to want to have as many options available to him/her to deal with your case during it's lifespan, and admitting fault before the lawsuit even begins will limit the lawyers options when it comes to negotiations and what tactics they can use in the courtroom.

[1] http://herokuclassaction.com/

incongruity• 13 years ago

In the grand scheme of things, this is why it's always better to defect in the simplified idea game of the prisoner's dilemma. Collaboration is rarely rewarded once lawyers get anywhere near an issue and opting to collaborate (i.e.: admit fault and try to fix it) more or less wins you no points in court, in the U.S.

Given that each possible liability is its own case and stands on its own merits, it pretty much matches the simplest case of the prisoner's dilemma perfectly, IMHO... and so here we are. Sadly.

rayiner3• 13 years ago

The US legal system isn't really set up any particular way with regard to first party admissions. It's just a statement that can be introduced into evidence like any other. Indeed, usually when they are treated specially it's to make them non-admissible. E.g. in an accident, your offer to pay the injured party's medical bills can't be introduced as evidence of fault.

The reason lawyers tell people to clam up is because jurors put great weight on what people say about their own actions. It's precisely the other edge of the sword of what you're talking about--a positive public statement can have a great effect on customers, but one that comes out wrong can have a very bad effect on jurors when the plaintiff brings it up in court.

seferphier• 13 years ago

Agree. Admitting liability in public before the trial started is very detrimental. After you admit liability, the only dispute left is how much you need to compensate the plaintiffs.

Just imagine being arrested. If you tell the police you are sorry- you are going to be found guilty. That is why all lawyers would tell you to remain silent until he arrives to the police station. The risk of admitting liability when talking is too big.

resu_nimda• 13 years ago

They have made public statements on their blog, and hopefully that will continue. In this situation, I think it is the right move to avoid getting involved with this shameless law firm trying to incite drama for their own gain.

redthrowaway2• 13 years ago

This seems like precisely the type of situation class action lawsuits are intended to deal with: a company defrauded a lot of its customers in the same way. It makes more sense for all parties that the case be tried as one than to hear each claimant individually.

Adaptive• 13 years ago

Sadly, the US legal system does not reward heartfelt, honest statements, particularly in these situations.

Which is why fine print almost always makes even the nicest company sound like a jerk.

anigbrowl• 13 years ago

There's a middle ground; it is better to fess up and try to square things with your customers in a situation where you've done wrong, but at the same time you don't want to be exploited by unscrupulous plaintiffs in turn. Most of those customers just want what they paid for or to recover their actual losses (what are called economic damages in law) but a few may see a lawsuit as an opportunity to enrich themselves. The court system is supposed to prevent that, and overall it works moderately well, but once papers are filed it's better for the arguments to take place in court rather than outside.

danso• 13 years ago

> Yes, admitting wrongdoing may open you up to legal liability. But it seems to me that, in the long run, it's far better to take your licks and own up to your mistakes than to hide behind the shield of pending litigation.

This is an admirable, well-intentioned perspective, but it is one that won't net you much benefit, not even in the long run.

Filing legal papers -- or, in this case, raising the banner for a class-action -- is serious business. By the time it happens, the time for defusing the situation and calling a truce is long past. The only opinion that matters is that of the judge and/or jury. And let's face it, being the loser in court is going to have far more impact on the public's opinion of the defendant than any honest-from-the-heart statement or outreach.

So with that said, there is nothing of substance that the defendant can say that will help them in court. The opposing attorney is not going to say, "Well, I was going to grill you, but I've had a change of heart ever since reading that impassioned, honest, salt-of-the-earth statement you released pre-trial". The judge won't care either. And the jurors aren't supposed to even be aware of it.

I think a civil trial is not much different than the situation portrayed in the famous "Don't talk to police" video:

http://www.youtube.com/watch?v=6wXkI4t7nuc

The people you're facing in court don't care that you've made an effort, post-lawsuit-filing, to be upfront and friendly. Anything you say will not help you, it can only hurt you. In this case, Heroku's reticence in speaking their mind outside of court is the right one.

OGinparadise4• 13 years ago

This is normal, once legal threats are issued "we can't comment on legal issues...but customers are #1...blah blah" is the standard answer.

A mea culpa right now might haunt you in the long term.

citricsquid• 13 years ago

If the decisions made by Heroku to misrepresent the product were made prior to the Salesforce acquisition (December 2010) do the founders of Heroku have any liability? Would this sort of thing be a consideration with an acquisition?

chc2• 13 years ago

I don't believe any decision was made to misrepresent the product. The product just evolved and the product documentation didn't. The end result is the same — the product was advertised with inaccurate claims — but this appears to be more a case of neglect than a deliberate misrepresentation.

jacques_chester11• 13 years ago

It's not a criminal matter. Mens rea -- "guilty conscience", the intention to act unlawfully -- is a component of crime. But it's not necessary for lots of civil legal matters.

For example, if we both enter into a contract, you can hold me to my end. If I fail to hold up my end without intending to fail, then you can still insist that I make right or seek remedies for my failure.

(IANAL, TINLA).

drucken• 13 years ago

It would have been an issue for the acquirer before or during acquisition, if it were present at the time, of course. Some, perhaps many, investors into startups also insist on liability insurance.

But the founders of Heroku have no more liability than the company in control of its assets, which in almost all cases would be some sort of Limited Liability vehicle.

pyre• 13 years ago

I think the parent was asking if SalesForce could pursue the Heroku founders for some sort of liability (i.e. misrepresenting the company during the purchase).

eli2• 13 years ago

IANAL, but my understanding is that threatening a class action lawsuit is easy, but actually bringing an action to court is extremely difficult. And (at least to me) this seems like an awfully weak case -- I don't see any obvious or intentional fraud.

The whole thing seems strange to me. I don't really understand what Rap Genius hopes to gain here. If Heroku wasn't providing enough performance for their money (regardless of technical cause), then why did they stick around so long?

Would Rap Genius still have sued if the documentation had been 100% correct and instead the problem was just plain old slow I/O on Heroku's side?

niggler• 13 years ago

Ianal stands for I am not a lawyer, so "a lawyer" in "IANAL a lawyer" is redundant.

eli2• 13 years ago

Hah, fixed. Sometimes my fingers get ahead of my brain when typing quickly.

tomlemon• 13 years ago

> If Heroku wasn't providing enough performance for their money (regardless of technical cause), then why did they stick around so long?

We didn't know how bad the performance was because Heroku's tools (logs and New Relic) reported incorrect performance data (i.e., they said requests weren't queuing when they were actually spending a ton of time queuing)

EDIT:

> Would Rap Genius still have sued if the documentation had been 100% correct and instead the problem was just plain old slow I/O on Heroku's side?

To be clear, we have no official association with http://herokuclassaction.com/ or the lawyer behind it – I actually found out about the site from the article's author when he interviewed me for the story.

(But I do think Heroku owes its customers a refund)

jaggederest2• 13 years ago

New Relic isn't Heroku's tool, man. It's like a car dealership selling you a Pioneer stereo.

swensel• 13 years ago

I don't think the issues with New Relic was necessarily Heroku's problem. Sure, Heroku did have some outdated info on their website and their logs may not be fully accurate, but I personally don't think that they should provide refunds for that. Solving the problems that have been raised would be a better service to their customers. Providing refunds probably doesn't help Heroku create a better service which is what I think will matter long term.

arihant• 13 years ago

I think after reading the article, it is pretty clear they did not notice the routing issues until they noticed slowed down performance and till their consumers complained. That makes perfect sense, you don't really dig deep into routing until you hit a bottleneck, specially when you are unable to monitor it, and specially when your provider is lying to you about how they route.

iends• 13 years ago

Several people have posted support logs that Heroku knew about the issue long before.

kapilkale• 13 years ago

Rap Genius isn't suing.

potatolicious• 13 years ago

> "I don't really understand what Rap Genius hopes to gain here."

Rap Genius isn't the ones doing the suing. Rap Genius wrote about it, which caused Heroku to come clean. I think Rap Genius already got what they hoped to get (i.e., exposure, official comment, etc).

> "I don't see any obvious or intentional fraud."

You don't need intent to be sued - you only need to have caused damages to the plaintiffs. What Heroku has effectively done was:

1 - misrepresented their product in a substantial way.

2 - "overcharged" their customers substantially by giving them less performance than they paid for.

Both of these are material damages, and there's a more than reasonable case for Heroku's customers to recover this money.

This isn't a criminal case here, there doesn't even have to be fraud (as defined in the criminal law context).

rayiner3• 13 years ago

A civil suit doesn't mean "you're a bad person." It means "we're not square." If you sold something with certain parameters and delivered less, even if you did it on accident you're not square with your customers, and even if your customer didn't notice you're not square with them.

jacques_chester11• 13 years ago

> I don't see any obvious or intentional fraud.

Fraud is usually a criminal matter.

This is a civil matter. Probably under torts law, especially if there's anything in California/US torts law that resembles Australia's tort of "misleading and deceptive conduct".

Fraud and torts are different. And to forestall the next obvious point: torts and contracts are different.

> I don't really understand what Rap Genius hopes to gain here.

A refund.

Heroku's selling pitch is linear scalability. Rent 1 dyno, get ~ (1 dyno / Σ dynos) improvement in performance. Rap Genius demonstrated that they were getting exponentially worse increments of added performance per dyno added.

> If Heroku wasn't providing enough performance for their money (regardless of technical cause), then why did they stick around so long?

This is orthogonal to the question of whether Heroku acted in a legal way.

> Would Rap Genius still have sued if the documentation had been 100% correct and instead the problem was just plain old slow I/O on Heroku's side?

Courts don't worry their heads with strawman hypotheticals.

(IANAL, TINLA)

LAMike• 13 years ago

On one hand the Rap Genius expose seemed like they got duped and mislead, but on the other hand something about this lawyer mentioned in the article comes off as ambulance-chase-y to me.

lancefisher2• 13 years ago

It's not really ambulance chasing if all he's after is a partial refund.

From the article:

As for Lehman, he wants some of the $20,000 his company pays Heroku each month returned. “We deserve some sort of refund here,” he says.

EDIT: I re-read the article, and it's not even clear that Rap Genius is part of this. Looks like a lawyer trying to drum up support.

rayiner3• 13 years ago

Class action lawyering usually sound ambulance chasey, because any given person doesn't lose that much in a given instance and so the actual people harmed have little incentive to sue.

At the same time, it's an edge case in the economy. If you harm a large number of people a small amount, they won't sue you to recover for their injury.

dubcanada• 13 years ago

http://www.wired.com/wiredenterprise/2013/03/hieroku/

HIEROKU!?

Ignoring the funny url. I doubt this will go anywhere besides in the domain expiry list.

lancefisher2• 13 years ago

From the article, Heroku says, "The problem affected an older version of Heroku’s service, he said, and those who use a new version of the service would not have these issues."

As far as I understand that is not really the case. The newer version is the Cedar stack which supports concurrent apps where Bamboo only supports non-concurrent apps. This would reduce the problem with random routing, but not eliminate it.

randall• 13 years ago

The problem is not that they provide random routing, the problem is they said in their documentation that they provided intelligent routing, a design choice they backed off of when they switched stacks.

Customers still have a problem, but Heroku doesn't from a legal / class action-y perspective.

bridgeyman• 13 years ago

In MIT's Mathematics for Computer Scientists course (6.042), I remember learning from Tom Leighton that in certain cases randomly distributing jobs actually IS the optimal solution to load balancing. This technique was somehow associated with Akamai, Professor Leighton's company.

Maybe I am misremembering...

w-ll• 13 years ago

I dont know, but I would assume it has to deal with the fact you can't predict how long a job will take to complete, and a random selection would have more resources-more free-more often, verses FIFO where a group of really large tasks could halt everything.

kordless• 13 years ago

That's exactly correct. Given jobs cleared out quick, that wouldn't necessarily be the case.

manmal• 13 years ago

And what if they turned the whole thing into a request queue and workers get the "messages" as they become available? To ensure there are no timeouts, they could define a max time a request may stay in the queue, and then take it out and handle it with emergency workers or kill it?

ww520• 13 years ago

I think the matter is if random distribution is what is documented, no one will raise an eyebrows. I believe the Heroku claimed intelligent routing based on server loads to route traffic to idle servers. That threw people off.

jonknee2• 13 years ago

Not only that, but they charged a premium based on their false claims. I'm surprised it took this long for legal rumblings to come around.

mkmkmmmmm• 13 years ago

I think you are supposed to take least busy of two random nodes.

rdl5• 13 years ago

Heh, nothing will make people go back to taking Heroku's side faster than a class action lawsuit on the other side. This would be an awesome form of submarine marketing if they were actually behind it :)

chubot2• 13 years ago

Um, why? I was one of many plaintiffs in a class action lawsuit against EA (which was settled in our favor). That did not help EA's image at all :)

It looks pretty bad when your customers complain about problems that are hidden by your own monitoring tools, and then you tell them to buy more crap to fix it! It's not a small amount either -- as mentioned the queuing time was larger than the request time. And on top of it you're paying out the nose for this crappy monitoring tool.

rdl5• 13 years ago

Even people who have been somewhat screwed by this issue with Heroku tend to love Heroku; Salesforce could make this right with generous payouts to affected people and 2-6mo of engineering to offer a better load balancer as a service (Actually, I think I could build this as a third party provider, even)

EA is basically a lost cause; their business model and culture is the problem. Having better sacks at Dachau wouldn't have helped the Nazi image, either (to auto-Godwin)

ozgune• 13 years ago

While I agree this sways me to Heroku's side, I don't think this is their marketing ploy.

Rap Genius says they aren't involved either, so this is probably a single-person law firm looking to create some buzz for himself. He should threaten to sue Amazon next for all the times they haven't held their end of the bargain. That'll get him a ton of publicity.

rdl5• 13 years ago

Yeah, I didn't mean to seriously suggest that it was the case, just that it would be funny.

scott_meade• 13 years ago

"[The attorney who set up this site] has spoken with one customer, who has since backed off. “That being said, I am in the process of drafting a complaint, in case their feet warm up,” he told us."

vertis• 13 years ago

It really disgusts me how quick people are to bite the hand that feeds them. Heroku is not perfect, but they give a whole lot more value than they charge for.

As unlikely as it is, I hope that Heroku terminates the account of anyone that joins the class action against them.

jonknee2• 13 years ago

> As unlikely as it is, I hope that Heroku terminates the account of anyone that joins the class action against them.

It would be hard to give worse legal advice than this.

jrockway4• 13 years ago

Oh come on, it's easy! "In my opinion as a lawyer, you should release radioactive bees into the refrigerators of your customers."

Now that's bad legal advice.

Karunamon5• 13 years ago

>It really disgusts me how quick people are to bite the hand that feeds them.

Really? Because it really disgusts me when a company:

    * Misrepresents their product (saying it's A when it's actually B)
    * Privately acknowledges this misrepresentation under the radar
    * Makes large and ridiculous amounts of bank on this misrepresentation (since you have to throw more dynos at the app to mitigate the lag caused by the misrouting.. and Heroku may be many things, but economical it isn't)
    * And on that note, provides very expensive monitoring and performance analysis tools that further perpetuate the misrepresentation!
    * And then force you to resort to litigation to get some portion of this huge mis-spent investment refunded. (Looking at the suit, this is all that's being asked for)

This wasn't a mistake on Heroku's part, this was calculated. And even if you accept that it was a mistake, the fact of the matter is that their negligence in this matter made them a hell of a lot of money.

This is definitely grounds for a suit.

res0nat0r6• 13 years ago

By misrepresents you mean that the end users didn't exactly understand how underlying architecture worked, didn't like the performance, but kept using the service anyway. If every customer I dealt with while working on the EC2 platform could sue because they didn't understand how every technical detail of AWS worked and they couldn't optimize for it...well there would be more lawyers employed by Jeff Bezos than engineers.

This lawsuit will go nowhere and it is sad that such a supposedly tech savvy crowd wants to bring out the pitchforks so fast. But I guess since Heroku has a lot of cash they are now the enemy since they've made it and they should be taken down.

jacques_chester11• 13 years ago

I guess I saw this coming:

http://news.ycombinator.com/item?id=5237920

http://news.ycombinator.com/item?id=5241686

Really, though, given how much money Heroku charge and how much money Salesforce.com would be able to stump up in a settlement or judgement ... it was inevitable that a fast-moving law firm would jump on this.

jrockway4• 13 years ago

TLDR: RapGenius (YCS11), in a painful attempt to remain relevant, decides to sue their web host for running their slow code slowly.

mikeryan• 13 years ago

If you're going to tldr; an article you should really read it. Rap genius isn't suing. No one, in fact, is actually yet suing.

jrockway4• 13 years ago

The first line in the article "Rap Genius founder Tom Lehman says Heroku misled him and cost his company money."

I have to imagine they're involved based on the number of articles I've read about them over the last week. Their own article, tech blogs interviewing them about the article, and now the founder's picture at the top of this article.

goronbjorn• 13 years ago

RapGenius just recently raised $15m from Andreessen Horowitz; they hardly need to make 'painful attempt[s] to remain relevant.'

And they're not even involved with this lawsuit.

piggity• 13 years ago

Only lawyers think it's a good idea to deploy lawyers into the routing mesh.

raverbashing2• 13 years ago

But you can bet money will be intelligently routed to their pockets

raverbashing2• 13 years ago

I think the money for this class action lawsuit would be better used if it would go for a sysadmin/devops person to actually deploy the systems involved.

"Bait and switch"? If a couple of Dynos is not enough for your load, you should be moving to self-managed hosts

DigitalSea• 13 years ago

At the end of the day Heroku are in the wrong here. Performance issues which Heroku basically wouldn't admit until they were outed about them meant that companies would have invested tonnes of time and effort which in turn equals money on trying to fix issues that were caused by Heroku the whole time.

bascule• 13 years ago

I'm sorry, did I miss something since aphyr's analysis? http://aphyr.com/posts/278-timelike-2-everything-fails-all-t...

Why is Heroku "in the wrong" exactly?

Coding Cloud Heroku May Face Lawsuit Over Bait-and-Switch Claims

Related Stories