DoS attacks that took down big game sites abused Web’s time-synch protocol

http://arstechnica.com/security/2014/01/dos-attacks-that-took-down-big-game-sites-abused-webs-time-synch-protocol/

by RougeFemme • 13 years ago

43 34 13 years ago

chris• 13 years ago

We (Weebly) had 18Gbps of UDP/123 (NTP) traffic sent our way on New Years eve -- definitely one of the larger attacks we've seen recently.

midas0074• 13 years ago

This sound like what brought down the GnuPG main site. Sounds like an Anonymous(TM)-type thing.

nradov• 13 years ago

So both DNS and NTP can be abused in amplification DoS attacks. The obvious question is: which other protocols can potentially be exploited the same way?

rwg• 13 years ago

Any protocol that rides in UDP and that will reply to an unsolicited n byte packet with a >n byte response. (But if the amplification factor is close to 1, then you might as well have your botnet just attack the target directly.)

Devices that speak SNMP and use "public" for their community could be used in an amplification attack. I don't know what the amplification factor would be, though, and I imagine there are loads more Internet-facing DNS and NTP servers than Internet-facing SNMP agents configured with a "public" community...

devicenull3• 13 years ago

SNMP is already being abused for this

codexon2• 13 years ago

The most common attacks we have seen come from DNS, SNMP, and Chargen. You can get up to a 50k byte packet from a 64 byte response, but this is highly variable.

SNMP is more prevalent than you might think, it is present and unsecured in many firewalls, routers, and printers.

midas0074• 13 years ago

It's funny you should mention this.

So I used to be the lead sysadmin for the largest profit-center dept at one of the most well known universities. I left due to pressure to degrade security of the network for credit card processing and cash registers. It wasn't a protest as much as wanting to keep some appearance of integrity.

On the plus side, the institution managed to deploy departmental firewalls and take Joe Sixpack's and Sally Sue's business desktops and servers off public IPs. If you wonder how many seconds it take to infect an unpatched windows box desktop, it's other the order of 60 s +- 25%. Yes, the desktop staff intentionally tried a few times just for kicks.

It's a cracker's paradise.... fast links and there are like 2 semi-security people. But their duties and tasks are so diffuse, it's almost impossible to catch anything except boxes.

Their organizational resistance even pushed away a couple of brilliant people you might have ran into at hope|ccc|bh.... Instituting a security officer and listening to them + letting them reach out to teach, influence are two different things.

ghshephard4• 13 years ago

Easy DOS to block upstream, thankfully. Time seems like the sort of thing that a CoLo should provided to it's tenants. GPS time is likely good enough for 99% of sites, and, the final 1% who really require super reliable time, and need to be concerned about GPS spoofing/outages, can install a local atomic clock in their cabinet.

sargun2• 13 years ago

How would you block it? Any packet with the source port of NTP should be dropped?

No. ISPs should not get into the business of default block rules. This gets into all sorts of problems. At what point does a block become unreasonable? What if a tenant needs multiple sites in sync -- they must run NTP between them (or PTP) in order to determine their drift from one another.

If you really want upstream blocking of traffic, the best thing to look towards is BGP Flowspec (RFC5575)

ghshephard4• 13 years ago

"How would you block it? "

The same way all Upstream providers blocks a DDOS - by characterizing the DDOS flow, scrubbing, and dropping any packet which matches that pattern.

The idea here, is that during a DDOS, where you might get a bit more NTP drops than normal, none of the local users should be impacted if they had a quality local time source.

"What if a tenant needs multiple sites in sync"

That's why you have a quality time source. By definition, they are "correct" to within any tolerance that all but CERN physics experiments care about, and they don't use NTP anyways.

See: http://static.googleusercontent.com/media/research.google.co... for an example of using synchronized time globally, without reliance on running NTP between sites.

In particular, check out section 3, "TrueTime"

devicenull3• 13 years ago

no major providers that I'm aware of support flowspec. it's also juniper only currently

pktgen3• 13 years ago

Obviously such rules wouldn't automatically be in place; they'd have to explicitly be requested. And yes, it is a reasonable request to ask your upstream to add a few ACL lines.

midas0074• 13 years ago

For smaller webops shops, blocking ntp might not be possible.

For larger ones, having local strata0 timesources are an option. GPS gives you very precise, relativistic time for extremely low cost. (Bias: /me == Frmr Trimble employee.)

skymt• 13 years ago

Why are ISPs letting spoofed traffic leave their network? I don't work at that level, but I thought this was a solved problem.

codexon2• 13 years ago

1. Because it costs time and money.

2. It doesn't help yourself. So for many companies the cost can't be justified to the shareholders.

sargun2• 13 years ago

I imagine you're suggesting most ISPs implement BCP38? A lot of organizations implement BCP38, but the overhead of BCP38 can actually be a lot, especially once you get far above the DSLAM / CMTS.

Not only is it not free, it is not straightforward. In some more complex network architectures, you don't even know what the sources of downstream packets might be (case: downstream has two transits, and they don't want to bring ingress traffic via transit B, because transit B has ridiculous pricing).

cma• 13 years ago

Its value is way higher than its overhead, but the overhead is paid by the guy who essentially doesn't benefit from it.

devicenull3• 13 years ago

because it's profitable? there are providers that allow this because it gets them a lot of sales.

mirashii• 13 years ago

As someone who was considering Black Lotus as a potential service for DDOS protection, they had a number of failures during this whole episode that make it hard to take them quite seriously.

First, and most importantly, they suffered many hours of downtime to many sites hosted on their service, including their own web page, and still to this day have not tweeted or given any public statement about the incident. A lot of Googling got me some responses to customers who complained in private with weak justification about it being their upstream provider's problem and not theirs. The fact that they now estimate that it was 28GBit/s in traffic makes me even less trusting of their service, given that much larger DDOSes are flying around.

If anyone has any other services that are competitive, other than Cloudflare, I'd be interested in hearing about them out of curiosity. It seems there should be more than one company that can do this well.

scurvy6• 13 years ago

defense.net and Prolexic come to mind before Black Lotus or CloudFlare. Black Lotus and CloudFlare are bottom of the bucket options compared to defense and Prolexic.

Keep in mind that Prolexic was recently acquired by Akamai, so prices will skyrocket soon.

mjn• 13 years ago

It only works for certain kinds of sites, but if it's a site you can host on third-party hosting (as opposed to needing DDOS protection of your own network or VPS), Nearlyfreespeech.net has a pretty good track record with mitigating DDOS on sites they host. They also have a strong ethos of not blaming the customer for "provoking" the attack, or terminating them for the inconvenience.

pktgen3• 13 years ago

That's how Black Lotus does business. It's like the time when they were caught spamming and were made to look like complete fools on WHT.

p1mrx• 13 years ago

The Internet is not the Web, and NTP has nothing to do with the latter.

PaulRobinson• 13 years ago

The Web != The Internet.

NTP is not "the Web's time protocol".

Really fed up with tech journalism these days assuming that nobody has quite finished the first few chapters of the "Networking for Dummies" book. Massive gap in the market for informed quality tech journalism.

DoS attacks that took down big game sites abused Web’s time-synch protocol

Related Stories