Cloudflare Attack Post Mortem; Apparent Google Apps/Gmail Vulnerability

http://blog.cloudflare.com/post-mortem-todays-attack-apparent-google-app

by demonfly77 • 14 years ago

118 63 14 years ago

blog.cloudflare.com

demonfly772• 14 years ago

this is the tweet where CosmoTheGod states that UGNazi hacked Cloudflare and 4chan https://twitter.com/CosmoTheGod/status/208656791708508160

lomegor• 14 years ago

This is really important news. It was mostly social engineering hacking, but it seems to have been very clever. It's important to notice that it wasn't a flaw in two factor authentication itself, but on the recovery options.

I really hate the recovery options. I don't know what they are there. If I forgot my password, my recovery e-mail, I don't want to answer questions to be sure it was me. Who knows how this exactly worked, but it sure seems to be a problem with that logic.

joelhaasnoot• 14 years ago

So it seems that there was a bug in the two factor authentication affecting "some accounts" during the recovery procedure (see the update to the article). Slightly worrying that the two factor authentication can be bypassed when resetting a password.

boundlessdreamz3• 14 years ago

But even after resetting the password, two factor authentication is needed to log in right? How was that bypassed?

quarterto• 14 years ago

WHERE IS YOUR GOD NOW, JEFF? [1]

[1] http://www.codinghorror.com/blog/2012/04/make-your-email-hac...

stephengillie2• 14 years ago

If only that admin had followed this suggestion for his personal account...

stack0v3erfl0w• 14 years ago

Have you read the article ? the 2 factor authentication wasn't compromised, it was mostly a social engineering attack. Even if it was the 2 factor authentication that was compromised, how would this be Jeff's fault ?

eastdakota• 14 years ago

There was a flaw in the system that Google uses to allow the transfer of control from an account. Two-factor authentication wasn't compromised itself, but the attacker was able to bypass it to access it. That flaw, they tell us, has since been patched on their end.

citricsquid4• 14 years ago

Cloudflare supports hacking group by providing them services -> gets hacked by them. Reminds me very much of what happens in every episode of 24, the terrorists kill their accomplices after they're finished with them, it never works out for anyone involved.

daeken• 14 years ago

Which "hacking group" was supported by Cloudflare?

redouane• 14 years ago

lulzsec

citricsquid4• 14 years ago

Lulzsec was the most prominent, they used Cloudflare to keep their sites online, Cloudflare used it as a marketing tool, here's a blog post about it: http://blog.cloudflare.com/58611873

tedivm• 14 years ago

Cloudflare doesn't necessarily "support" hacker groups- although their CEO did a great talk at Defcon last year about keeping Lulsec up (which I consider more of an activist group than a hacker group, personally). However, they do very little to keep their network safe- they have no dedicated abuse team (or, if they do, they're impossible to contact directly), are slow to response to abuse requests (if they respond at all), and even when they do respond take only the most basic steps to filter the issues (which takes literally minutes for their account holders to work around, and they don't actually disable the accounts).

Dozens, if not hundreds, of hacker and malware related sites use Cloudflare to mask their origin and essentially use the other shared sites as hostages. If you block one of the malicious sites then you also end up blocking legitimate sites, and since Cloudflare goes out of their way to not take down sites (even those hosting malicious programs- including the drive by exploits that install them) it makes it very difficult to trace the malware back to it's source or actually take it down and stop people from getting infected.

Bootvis• 14 years ago

Lulzsec and the people that did this are not necessarily the same.

I find it a bit strange to lump a hosting provider together with terrorists accomplices.

News report about CloudFare and Lulzsec: http://www.it-networks.org/2012/03/01/how-cloudflare-kept-lu...

citricsquid4• 14 years ago

I know the groups are different, my observation was that Cloudflare supports these groups by providing them with services and in return they themselves become a target of such groups. My terrorist comment wasn't comparing them to terrorists, I guess a better point would have been "no honor among thieves". Cloudflare support these sort of groups and in return they are targeted.

ashconnor• 14 years ago

The headline of this submission is misleading.

tl;dr; Hacker used social engineering to add a secondary recovery email address to the victims Google account.

2-factor wasn't even enabled on the victims personal account but the hacker didn't need a 2-factor code to reset the password on the victims company account.

From the article:

Google reports that they discovered a "subtle flaw affecting not 2-step verification itself, but the account recovery flow for some accounts. We've now blocked that attack vector to prevent further abuse."

stephengillie2• 14 years ago

The title of the article doesn't even mention 2-factor: Post Mortem: Today's Attack; Apparent Google Apps/Gmail Vulnerability; and How to Protect Yourself

Don't commingle business and personal accounts.

Strom2• 14 years ago

It's not misleading. It should not be possible to recover your way past 2-factor auth, which was enabled for the company account.

Havoc• 14 years ago

Thank you. I was skeptical about that headline but didn't feel like reading that wall of text just to make sure its BS.

kinleyd• 14 years ago

Yes, it qualifies as social engineering rather than cracking. The original title is more appropriate.

eli2• 14 years ago

How is that social engineering? It doesn't seem like human behavior played any role in this at all. The hacker interacted only with automated systems.

This is a security bug that was exploited. It should not be possible to reset a password without the 2-factor auth code, period. If that is possible, then that the 2 factor auth is broken.

danielpal• 14 years ago

When people say 2-factor auth code is broken, it implies there was an issue with the 2-factor authentication code itself. That would mean things like RSA and other Token based authentication might be at risk. So no 2-factor auth is not broken. Authentication in Google is broken.

-disclaimer: I am the founder of Authy.com.

StavrosK2• 14 years ago

If I understand it correctly, the attacker managed to reset the password by adding a secondary, fraudulent email account as a recovery option, and Google just logged them in after the password was reset, not requiring a token from the 2-factor system.

That's quite the attack.

demonfly772• 14 years ago

as stated on Softpedia, UGNazi got access to all Cloudflare customers' IPs and IDs, and they will sell these informations http://news.softpedia.com/news/CloudFlare-Reports-Breach-UGN...

rdl3• 14 years ago

There are a lot of good reasons not to use gmail or google hosted apps (performance, especially, but apparently also security). Two factor auth as google implements it goes partway, but email really is the skeleton key to all accounts. I'm not sure how meaningful Google Authenticator is as two factor auth when you're actually using the iPhone or iPad to access mail much of the time. Google Authenticator protects against untargeted mass attacks, but if you are specifically a target, it's a pretty mediocre two factor solution. I think it's fair to consider administrators of any major SaaS app to be specific targets.

Unfortunately, there really isn't a great alternative. The next best thing I've found is to just run your own mail server, which is pretty obnoxious if you have any non-technical users to support. Kerio is the thing which most Mac shops I know use (since people want directory services and the other stuff you can't easily implement just from open source). Yet, there's better search on server on gmail than from iOS to IMAP (i.e. Kerio), so that's a sacrifice.

I kind of hate Google for being the Craigslist of email -- a crappy product at $50/yr which sets the maximum price people are willing to pay below the point where anyone can offer a less crappy product. More like $250-$500/yr/mailbox would be a supportable price for real email.

tommi• 14 years ago

Please define "real email" and what justifies it to cost $250-$500/yr/mailbox in your opinion.

Terretta• 14 years ago

If you're questioning the grandparent's notion that Gmail is "crappy" compared to expensive email products, I'd agree with you.

For me, the answer to what justifies a cost for email is deliverability. Of course, having recipients receive your email promptly, and receiving theirs promptly. But more importantly, receiving next to zero spam in my actual inbox, and misclassifying absolutely zero ham in my spam folder.

The only solution I've found that accomplishes this at any price is GMail / Google Apps.

Not only does it easily keep 12 - 18 year old email addresses that have been on usenet spam free, it happily handled over 3000 spam per second per box while letting through valid emails for sex.com. The owners could actually use their addresses on Google Apps. They hadn't been able to on email systems costing 4 and even 5 figures. Refreshing the spam label while seeing the inbox stay empty, day after day and month after month, was astonishing.

namidark• 14 years ago

Look at Exchange and Zimbra

sauteedbiscuits2• 14 years ago

The biggest weakness's to 2 factor auth are:

1: You can remove 2 factor auth without having to enter the auth code. So if your computer gets a virus, someone can just remove 2 factor auth via your browser

2: The "app specific passwords" are full on backdoors. If one of your app specific passwords gets hijacked, someone can login to your gmail with it and then remove the 2 factor auth pass (See above)

* http://productforums.google.com/forum/#!category-topic/mobil...

reedlaw• 14 years ago

Aren't app specific passwords simply single-factor auth tokens, thus defeating the purpose of using two factor auth? Wouldn't a sufficiently complex pass-phrase be just as secure?

harshreality• 14 years ago

If 2-factor is used simply to log in, then any attacker who targets your phone/tablet to get your totp secrets from google authenticator could compromise rsa token protected logins as well. They'd only have to wait until you log in legitimately and compromise your account at that point.

Real security against compromise of the device you're using to access a service, whether the 2nd factor is a hardware token or generated from a seed stored on an android/ios device, would require making every significant action, not just logins, re-prompt for a token. I haven't seen many systems that do that, because it harms usability.

I'm not disputing that there are cases where RSA tokens or other hardware solutions make sense, but the physical burden of more than a few hardware tokens would be too much for normal people, even if cost were no object. 2-factor totp/hotp oath auth is far better than 1-factor auth, and presents minimal annoyance to people who already have their smartphone with them 24/7. It can be deployed on every saas website with minimal additional burden to users.

A compromised android/ios device that you use to login to services is, as outlined above, pretty close to game over even if you have a separate hardware token. An attacker may not be able to maintain access, but the account information and various settings can be compromised in an instant. 2-factor of any sort pretty much solves the password reuse problem, and also the problem with individual service passwords getting compromised somehow in a way other than an ongoing compromise of the user's machine(s).

sauteedbiscuits2• 14 years ago

"We have senior contacts at Google who we worked with in order to regain control of the Google Apps accounts"

Boy, its nice they get that privilege. I love gmail, but god help me the day I lose access to my account- I can only imagine the support whack-a-mole us mortals would have to go through to get any resolution.

Anyone remember the "thomas monopoly" story from a while back where gmail disabled the guy for weeks with no comment and the dude had to contact every person in the world to get someone to look at it? (And then they reinstated his account)

yaix• 14 years ago

My first though too.

They got great support, because they have a name. Any normal Gmail user would receive five automated reply emails from Goog, at most.

I'd love to move my email somewhere else, but there is not really any alternative.

antidoh• 14 years ago

Here's a list of alternatives, and discussion.

http://www.emaildiscussions.com/showthread.php?t=64118

fluidcruft• 14 years ago

Google Apps accounts are billed and offer phone support, right? I don't think there's a mechanism to upgrade/migrate free Gmail accounts to Google Apps, though.

notatoad2• 14 years ago

Cloudflare isn't special. Any business (or individual) who pays for google apps gets 24/7 email and phone support. I hate this perception that google has terrible customer support. If you want support, stop being a free user.

larrys• 14 years ago

Google having 24/7 email and phone support for google apps doesn't seem to jive with this statement in the parent:

"We have senior contacts at Google who we worked with"

Saying "senior contacts" seems to indicate that they have specific high level people that they personally know who were able to escalate beyond what the response would be for a typical user.

kolev2• 14 years ago

I friend of mine got his Gmail deactivated (as you know, without any explanation or means to dispute it). With it, hundreds of purchased Android apps got lost as well. As there isn't much you can do, after a week, he created a new account and had to repurchase most of the Android apps and switch all his accounts to it. Some services require access to your old email, so, he had to recreate some of those as well.

mmahemoff• 14 years ago

A potential security hole is very different from someone being locked out of their account (as annoying as that is). Most web companies by now have sound procedures in place to quickly escalate those reports.

And yes, realistically, a larger service like CloudFlare is going to get more attention than any individual. It might seem unfair to the individual, but overall, it's in everyone's interest when a situation like this arises.

druiid• 14 years ago

So on one side Cloudflare is saying that they just compromised a single account and on the other we have the hacker claiming that all data was compromised... I guess at that point the question becomes not whom is telling the truth, but what any Cloudflare users (myself included) should be doing right about now?

notatoad2• 14 years ago

Presumably after the hacker compromised the CEOs email account he had access to any account he wanted, as I interpret the blog post he was only able to use that access to compromise a single account before being locked out.

Nobody is necessarily lying.

runn1ng• 14 years ago

Just an information - the unnamed customer was 4chan.org, and, according to the group ... I think twitter, it was because they don't like My Little Pony fans, that have a board over there.

Stupid internet drama indeed.

veverkap• 14 years ago

Seems to me that there were two flaws evident here. One was Google's reset flow, which they have since (hopefully) corrected.

The other issue, that seemed to be glossed over, was that they were BCCing password reset emails to a compromisable account for support/debugging purposes. I understand the desire to be aware of fradulent submissions early, but we, as an industry, should be treating password reset emails (and tokens) like passwords. Do not store them in plain text.

I appreciate CloudFlare's transparency on this, but feel they missed an opportunity to address a flaw that probably exists in a large number of other apps.

FuzzyDunlop• 14 years ago

Copying the emails stood out to me as a ridiculously stupid thing to do. It was probably a quick and dirty solution to save coding up a way to monitor resets on the backend.

MPSimmons• 14 years ago

They say that they have disabled that now, so that's something.

wpietri• 14 years ago

The most interesting part to me is that somebody managed to get phone support for GMail.

ericabiz• 14 years ago

Google Apps for Business (a paid solution) offers phone support: http://www.google.com/apps/intl/en/business/upgrade.html#utm...

Since they were using their own domain name, that's likely what they were on.

lsh123• 14 years ago

The funny part - cloudflare still hosts DNS for UGNazi

   Domain Name: UGNAZI.COM
   Registrar: ENOM, INC.
   Whois Server: whois.enom.com
   Referral URL: http://www.enom.com
   Name Server: LEE.NS.CLOUDFLARE.COM
   Name Server: RUTH.NS.CLOUDFLARE.COM
   Status: clientTransferProhibited
   Updated Date: 29-may-2012
   Creation Date: 22-jan-2012
   Expiration Date: 22-jan-2013

Cloudflare Attack Post Mortem; Apparent Google Apps/Gmail Vulnerability

Related Stories