Reading Uber’s Internal Emails: Bug Bounty report worth $10K

http://blog.pentestnepal.tech/post/149985438982/reading-ubers-internal-emails-uber-bug-bounty

by vinnyglennon • 10 years ago

137 51 10 years ago

blog.pentestnepal.tech

tyingq5• 10 years ago

Ouch. No domain verification required by Sendgrid before allowing you to inject a hook that dumps email contents.

That's much broader than just Uber.

Edit: Yes, it's been fixed, but the fact that it existed for quite some time is still troubling. I'm also curious if the fix retroactively disabled any existing unverified hooks.

innoying4• 10 years ago

I believe they did retroactively search for accounts.

Source: I had a number of accounts banned when testing different iterations of this bug.

gizmo• 10 years ago

This looks like a massive security flaw on sendgrid's side. They should use DNS validation like everybody else to prove ownership of the subdomain.

wahnfrieden• 10 years ago

From the article:

Also at the moment of writing this bug, it has come to my attention that SendGrid has added extra verification which forces you to have a verified domain before adding a inbound parse webhook.

innoying4• 10 years ago

Technically they did employ some DNS validation. You had to setup a MX record to point to SendGrid before you could add the domain to your account. The problem was in order to send emails from a domain you had to add the same MX record. If you never setup the receiving end as well (on SendGrid) you were vulnerable to a takeover from another SendGrid account.

ransom1538• 10 years ago

Totally just curious. The law for "exceeds authorized access" is 20 years in prison. I think uber has a bug fixing program. Maybe sendgrid does? Do the DNS carriers? Does his ISP? I read the law. If one of these companies refutes access - this guy is facing 20 years [1]? W(why)TF do people do this? The US government is notoriously creative in these prosecutions [3]. Companies refute access all the time to not look like idiots EVEN WITH a bounty program. [2]. If I came across this I wouldn't be blogging about it for 10k?

[1] "(2) intentionally accesses a computer without authorization or exceeds authorized access, and thereby obtains—(C) information from any protected computer;" (C) except as provided in subparagraphs (E) and (F), a fine under this title, imprisonment for not more than 20 years, or both, in the case of—"

https://en.wikipedia.org/wiki/Computer_Fraud_and_Abuse_Act

[2] http://motherboard.vice.com/blog/facebook-is-refusing-to-pay...

[3] https://en.wikipedia.org/wiki/Aaron_Swartz

tyingq5• 10 years ago

Uber's bug bounty program seems like explicit permission to me. The twist here, as you mention, is that it is sendgrid's infrastructure. Chasing bug bounties does seem a bit risky in this "cloud" era where it's not just one entity running the targeted service.

beejiu• 10 years ago

It is not listed as an in-scope domain on Uber's bug bounty policy. You would have to argue that the access was unintentional - that you believed you had authorization to access the server from Uber and that Uber had the ability to assign that authorization on behalf of SendGrid.

bboreham• 10 years ago

Author appears to be in Nepal, which has no extradition treaty with the USA.

hartator• 10 years ago

It's just me, or $10k is far from being generous?

tyingq5• 10 years ago

The bit that's not generous to me is that Sendgrid, who made the mistake, is paying $0.

Hydraulix9895• 10 years ago

I thought so, too. Uber pays its full time employees some of the highest salaries in the Valley. This exploit could have easily fetched 10x the money on the black market.

innoying4• 10 years ago

Why does this comment appear on every bug bounty HN thread? Straight from the horse's mouth [0]:

  The black market is very unlikely to be a place you could sell a bug in a specific 
  website or service. It is not “worth millions”. Please stop repeating this.

[0] - https://medium.com/@collingreene/to-the-bounty-hunters-9259b...

Hydraulix9895• 10 years ago

What a cute strawman (and completely incorrect strawman from the wrong horse at that, this is Trumpist drivel from the guy who runs the actual bug bounty who of course has some acute rationalizations for underpaying for some of the most intricate technical work in the industry).

Nobody said it was "worth millions" but I have second-degree connections in Scandinavia that would pay 10x, like I said ($100,000).

@collingreene doesn't sound too familiar with these rather-illicit organizations, he strikes me as a product manager type person with a loud voice, not someone who actually has found and sold zero-days before. Maybe he doesn't have the technical acumen to do so, but hey, I'm not one to judge.

It's hard to establish proof that the market value on the black market is, in fact, much higher given that it is the black market (you're not going to find these people on Medium); However, one public example of this is the leaked Stuxnet details showing similarly high 5-digit prices for zero-days.

This isn't a specific bug either (it wasn't "oh, the log files for that one UberEATS micro-service were visible"), this flaw allows you to intercept the emails of pretty much any single one of SendGrid's clients. Imagine the damage someone could do with that, had it gotten into the wrong hands. Only $10k, what an insult.

EDIT: Upon further examination, it turns out that said author also contradicts himself and corroborates my own argument:

https://medium.com/@collingreene/why-product-security-is-har...

Primary source: https://www.wired.com/2016/09/top-shelf-iphone-hack-now-goes...

ComputerGuru• 10 years ago

The last time there was a SendGrid article on here, the feedback from the community was far from kind [0]. I again re-iterate that SendGrid has no business sending emails [1].

[0] https://news.ycombinator.com/item?id=12142728 [1]: https://news.ycombinator.com/item?id=12145019

dinersanddrives• 10 years ago

This! I actually started using them when they helped me out a hackathon but switched to a more dev friendly service.

mikesea• 10 years ago

Someone reported this same vulnerability to us via HackerOne months ago. We worked with Sendgrid support to re-claim the domain and they said they were urgently working to fix the issue, or not.

Edit: just saw this post was from September. Author probably made thousands in rewards circulating this vulnerability.

innoying4• 10 years ago

I do not believe the author circulated this report to multiple companies, however once it was made public a number of other reporters in the community did and continued to iterate on it until SendGrid fixed the issues.

Source: I am a member of said community: https://bugcrowd.com/bored-engineer, https://hackerone.com/bored-engineer, etc

ndaiger• 10 years ago

Sendgrid allowed attackers to social engineer control of my company's account and intercept password resets, despite an explicit warning from us a week prior (we received a chat transcript of the failed attempt and let them know that it was not us and someone was actively trying to social engineer access to our account).

Then they had the gall to try to convince me on the phone that it must have been my fault (after our blog post about it blew up).

Needless to say, I think they are terrible.

Previous HN discussion: https://news.ycombinator.com/item?id=7476836

andrewvc• 10 years ago

As a former customer (it has admittedly been a few years) I'm not surprised. Sendgrid's entire business is based on price. Emails are one of the few costs that don't really scale that well. They cost a lot more than people think they should. When I list did a cost analysis they beat out a lot of other providers.

I haven't used them in years, so I'm not sure if they are still the low cost leader, but they are definitely the kind of thing where you get what you pay for. Their lack of investment in product and infrastructure showed for the years I used them, and their slowness in delivering updates was incredibly frustrating.

philliphaydon• 10 years ago

Who would you recommend then?

edoceo• 10 years ago

I was on the fence about leaving SG, now I'm for sure out, thanks.

skeletonjelly• 10 years ago

I think this was submitted because it was discovered on /r/programming

zaroth• 10 years ago

I found this write-up a bit confusing and hard to follow.

The vulnerability is that any SendGrid user could configure a webhook callback which would POST back all received emails for any domain which had its MX set to 'mx.sendgrid.net'. OP exploited this against Uber to receive copies of their emails.

Presumably there was no way to tell from one account that another account is web-hooking your email out from under you. So you have to wonder, if it's as easy as just typing the domain you want to listen in on.... who else was getting all their emails tapped this way?

From Sendgrid's documentation;

  Setup
  The following steps are required to begin
  parsing email:

  Point the MX Record of a Domain/Hostname or 
  Subdomain to mx.sendgrid.net

  Associate the Domain/Hostname and the URL in 
  the Parse API settings page.

Shocking omission by Sendgrid, where's their write-up and apology?

downandout• 10 years ago

The problem is on SendGrid's side, however I doubt they would have paid $10K for it. He was smart to take it to Uber, who is likely (by far) SendGrid's largest customer, and who certainly has much deeper pockets than SendGrid.

uraniumhacker3• 9 years ago

At the time of the exploit, there was only one check, the MX record. Also once I claimed the domain, another user cannot claim it from their account. So this could be exploited silently as well because the victimized company won't know about it for a while.

wongarsu• 10 years ago

Honest question: Do bug bounties with such low bounties really do more good than harm?

A bug bounty incentivizes people to look for bugs. But when you find an interesting bug like this you have the option of either having the possibility of making millions from the social engineering possibilities alone, or claiming the bug bounty and get $10k. Of course claiming the bounty is the moral thing to do, but some people with both the skill and motivation to do bug bounties are bound to have lower moral standards.

arkem3• 10 years ago

I think it's hard to monetize most bug bounty bugs.

There isn't really a market for most XSS, CSRF or even RCEs bugs for web properties. Getting the bounty payout for a bug from the owner is often the best deal available. I think the only exception to the no-market situation is browser RCEs and smart phone OS jailbreaks.

Hydraulix9895• 10 years ago

Right, and this one could have easily been monetized because it allows attackers to intercept and read any one of SendGrid's customer's private internal emails!

otakucode2• 10 years ago

It's a good question, but I think the answer is no [err, re-reading your question I mean yes... thought it asked whether they did more harm than good]. Consider that you have a population of developers. Some of them are willing to break the law and cause highly probable harm (if only monetary) to someone else. Some are not. Those who are willing are motivated exactly as much whether there is a bug bounty program or not. Those who are not willing are motivated some amount (whether small or large, it is nonzero) by the existence of a bug bounty program to have a look and to turn over the information rather than sell or use it.

So, on the balance, you are increasing motivation for people who are not willing to harm for cash while leaving the motivation of the willing unchanged. I would be tempted to argue that it is flat out unwise to NOT run a bug bounty program, although it would be much smarter to offer larger bounties. I could make an argument for bounties exceeding the projected amount the vulnerability would be worth on the black market, I think, but that's a different subject.

bdittmer2• 10 years ago

This is a lot like a bug I found in Heroku's system a few years ago. Basically, if someone doesn't claim the wildcard subdomain for their primary domain and has a wildcard SSL cert anyone could (can?) claim subdomains. A quick google search yielded hundreds of exploitable domains. At the time it seemed like a pretty big vector for phishing.

I have no idea if they fixed this and they gave me a t-shirt.

brandur• 10 years ago

I can't recall all the exact details, but there is some validation logic in place along the lines of "if there's a wildcard domain installed then newly added subdomains must be on the same account as the wildcard's owner". You could give it a shot, but I don't think this attack would work.

(I used to help maintain the system responsible for this, but don't work there anymore.)

bdittmer2• 10 years ago

This was probably 2012 or 2013. I'll try to find the write up I sent them, but this check definitely did not exist then

edit: here's what we sent to heroku https://gist.github.com/bdittmer/6461b7a5093acd7d6263

dzhiurgis• 10 years ago

they gave me a t-shirt.

I had same experience with Salesforce

joshmn• 10 years ago

Would have been worth more to SendGrid, don't you think?

abalone• 10 years ago

Looks like they found a similar "3rd party email service / subdomain MX record" exploit with Slack.[1] Although not so severe in Slack's case because it's a lesser used feature. Looks like Slack is using Mailgun.[2]

[1] http://blog.pentestnepal.tech/post/150381068912/how-i-snoope...

[2] http://mxtoolbox.com/SuperTool.aspx?action=mx%3aslack.com&ru...

foota• 10 years ago

Several months ago I reported a bug to a popular service which was allowing password reset & other emails to be retrieved with just a serial id from a service like sendgrid.

uraniumhacker3• 9 years ago

Hi thank you for the post. If anyone has question, feel free to reach out to me here or at uranium@uraniumhacker.tech. As a writer of this blog, I will be able to provide the feedback necessary and clear any misunderstanding/false positives regarding this.

Reading Uber’s Internal Emails: Bug Bounty report worth $10K

Related Stories