Just one bad packet can bring down a vulnerable DNS server thanks to DNSSEC

https://www.theregister.com/2024/02/13/dnssec_vulnerability_internet/

by tptacek • 2 years ago

155 176 2 years ago

tptacek42• 2 years ago

I had no hand in writing this headline.

No, but admit it, you felt a certain glee in submitting it... (which is not to say you're wrong about that, and I write that as someone who once penned a public standard mandating DNSSEC -- which just goes to show we all make mistakes...)

tptacek42• 2 years ago

I think the case against it, while extraordinarily strong, is actually pretty subtle, and I generally don't look at advocacy for it in the early days as a mistake at all. We learned a lot from the experience of trying to get it to work. I was a vocal advocate for key pinning, and I don't think I was mistaken to push for it, despite its ultimate non-viability and the success of the competing CT model.

teddyh17• 2 years ago

Note: can bring down a DNSSEC-validating DNS resolver. Not any authoritative DNS server.

Most people already use a centralized resolver like Google or Cloudflare anyway, and you can be sure those will be patched. People who use their ISP’s resolver might be disappointed if their ISP is slow to patch things, but what else is new?

systems_glitch2• 2 years ago

I bet it'll still manage to serve ad pages instead of NXDOMAIN, though. Actually that's what will get them to patch up.

wtallis3• 2 years ago

Isn't the ads-instead-of-NXDOMAIN attack the one thing DNSSEC is actually good at protecting against? So I would expect the guilty ISPs to avoid providing any form of DNSSEC support whatsoever, and thereby avoid consequences of any of DNSSEC's downsides.

quickthrower23• 2 years ago

That is quite a stretch to get to that “meh”. I don’t even use 1.1.1.1 etc. and I am a HNer.

icedchai• 2 years ago

Same. I've been running my own DNS servers (caching and authoritative) since the mid 90's, back when my home network was on an ISDN line. I still use BIND.

8organicbits• 2 years ago

Lots of people use 1.1.1.1 without knowing it. Firefox, for example, uses DoH by default. The default resolver for the US is Cloudflare.

https://support.mozilla.org/en-US/kb/firefox-dns-over-https#...

Chrome followed:

https://duo.com/decipher/google-makes-dns-over-https-default...

tentacleuno• 2 years ago

People who use their ISP’s resolver might be disappointed if their ISP is slow to patch things, but what else is new?

Bear in mind that if you're using a regular port 53 resolver like Google's, you might be using your ISP's resolver anyway -- a lot of ISP's hijack port 53 and redirect it to their own server(s) to reduce load. That's why DNS over TLS / HTTPS is typically recommended over unencrypted DNS.

ptk• 2 years ago

Do you have examples of ISPs that do this?

cma• 2 years ago

How much does that reduce load, isn't it fairly miniscule bandwidth compared with non-colocated video streaming etc.?

TacticalCoder2• 2 years ago

Most people already use a centralized resolver like Google or Cloudflare anyway, ...

This is HN. I run unbound on a Pi (mentioning the Pi I don't know if those running PiHole are running a vulnerable resolver or not). My Pi shall need patching when I get back from vacation!

PreInternet015• 2 years ago

At this point, someone should just file a high severity / high impact CVE against DNSSEC in its entirety, and ensure the box-tickers all across the world mandate its eradication.

While I'm sure (even stronger: I know) that at some point DNSSEC was a good idea, it's just a liability-slash-infrastructure-tax these days (a footgun at best, an additional DoS vector in most cases), and it's time for it to go.

99% of DNSSEC deployments come down to "I trust the entity I have outsourced my DNS management to with my private key management as well", which makes no sense whatsoever, and the reasoning behind the remaining 1% doesn't inspire much confidence either.

DoH satisfies the DNS privacy/security requirements of most Internet users just fine, and with or without DNSSEC, the management of the underlying infrastructure remains the same as it was before, i.e. "trust us"...

pornel3• 2 years ago

DNSSEC doesn't have encryption, and does nothing for end-user privacy, so that's not even an alternative to DoH or DNSCurve.

The only thing it deems a potential privacy issue is possibility of enumeration of subdomains, which is security by obscurity for the domain owner, from a time before certificate transparency logs ruined that anyway.

RegnisGnaw• 2 years ago

NSEC3 pretty much solves this issue.

jcranmer2• 2 years ago

from a time before certificate transparency logs ruined that anyway.

Isn't it possible under WebPKI rules to get an intermediate cert for all of your subdomains so that only the intermediate cert needs to get logged in certificate transparency? Or at the very least, you could use a wildcard underneath the domain you own...

JackSlateur4• 2 years ago

Hum, DNSSEC is highly useful tho

Both DNSSEC and DoT/DoH prevent hijack between the customer and the resolver

However, only DNSSEC prevents hijacks between the resolver and the authoritative servers

DNSSEC is the only way to trust a zone's content

Or maybe DoT/DoH shall be the only way to resolve things, and the DNS over UDP/TCP shall be deleted from the surface of the earth ?

ekr____3• 2 years ago

It's important to understand why one might care about trusting the zone's content.

From the perspective of something like a browser, the mapping from the domain name to the IP address (i.e., the A or AAAA records) need not be secured via DNS because the connection to the server is secured via TLS and the WebPKI. So DNSSEC isn't helping here, especially because the clients don't check DNSSEC, so it doesn't protect them from attack on the local network, which is one of the main loci. If the DNS server provides the wrong IP, this is mostly a DoS attack.

I am aware that the situation is somewhat different in email but I'm not sure how different really. Note that it is the case that DNSSEC could help secure ACME validation queries, but as tpacek observes, CT has greatly reduced the risk of misissuance.

Of course there are other mappings in the DNS, but as a general matter those are being designed under the assumption that the DNS is untrustworthy, due to the low level of DNSSEC deployment. For instance, if you look at the HTTPS RR, it's fine to put a key for Encrypted Client Hello in it because the resolver already knows the desired domain name, so lying about the key doesn't really help. However, we can't safely publish the server's public key to be used for TLS 1.3 early data ("Zero RTT priming") because that would require trusting the DNS. So, any features which require the DNS to be secure have the usual chicken-and-egg deployment problems (this is also to a great extent what happened with DANE).

Taking the opportunity to reference myself, the following longer writeups might be useful on this topic:

https://educatedguesswork.org/posts/dns-security-dnssec/ https://educatedguesswork.org/posts/dns-security-dane/

PreInternet015• 2 years ago

DNSSEC is highly useful tho

Not as it's deployed today. Since over 80% (conservative guesstimate) of the zones that most people care about are not signed, it's pretty much useless.

The 'evil ISP or IT MITMs my DNS traffic' scenario is much more effectively addressed with DoH (since it only requires clients and some resolvers to coordinate, not the entire Internet, and it looks like regular HTTPS, which implementers already understand how to deal with, as a bonus).

And the 'evil government redirects some or all zones' play is still very much possible even with DNSSEC, since, guess who ultimately controls the keys.

Even if you think that making the latter scenario more difficult to implement is worth it, getting more people to sign their zones is a losing battle, since they're very much likely to self-DoS themselves in the process, significantly reducing their enthusiasm...

mike_d4• 2 years ago

However, only DNSSEC prevents hijacks between the resolver and the authoritative servers

It does no such thing. An attacker on the resolver or between the resolver and authoritative is able to strip DNSSEC.

DNSCrypt was the solution to all the problems we had, but at the time DNS software vendors and operators also had a financial interest in passive monitoring so DNSSEC being clear on the wire won.

belorn2• 2 years ago

It sounds good as long we apply it to all outsourcing where security relies on a third-party. End-to-end encryption was never designed with the concept of clouds and anycast providers, including the whole idea of virtual servers. They all depend on "I trust the entity who owns this hardware" which can be any partner which the cloud provider deemed worthy.

codegeek• 2 years ago

I am not an expert in DNS but isnt' DNSSEC sort of like IPV6 when it comes to utility vs adoption vs overhead ? In other words, hasn't really lived up to the expectations or am I missing something ?

tptacek42• 2 years ago

It has massively underperformed expectations in a way that far outstrips any disappointment anyone could have in IPv6. IPv6 has adoption approaching 50%. If you have a modern ISP and a modern router, you might be using IPv6 right now without even noticing it. Meanwhile: less than 5% of the major US TLDs are signed, and an even smaller fraction of the "top domains" (given any list of top domains, like the Moz 500) are. The major prevailing use case for DNSSEC, which is DANE, has no browser support --- browsers actually tried it and removed that support.

teddyh17• 2 years ago

My usual alternative viewpoint whenever you try to claim that DNSSEC is dead:

From where I sit, I work at a registry and DNS server host (among other things) where about 40% of all our domains have DNSSEC (and that number is constantly climbing). Every conference I go to, and in every webinar, people seemingly always talk about DNSSEC and how usage is increasing.

From my perspective, your continuous claim that “nobody uses” DNSSEC is simply false. DNSSEC works, usage of DNSSEC is steadily increasing, and new protocols (like DANE) are starting to make use of DNSSEC for its features. Conversely, I only relatively rarely hear anything about MTA-STS.

(Originally written as a two year old comment of mine: <https://news.ycombinator.com/item?id=27837530>, but the details given have not changed substantially.)

tick_tock_tick• 2 years ago

At what percent adoption will you give up? It's been marching up non stop and with the inclusion of DNSSEC in FedRAMP requirements any hope of it being a "failure" has long since past.

pornel3• 2 years ago

DNSSEC struggles to justify its existence, because it's mostly useless without transport security, and for transport security we use HTTPS (TLS+WebPKI) that doesn't rely on DNSSEC. This leaves only few scenarios where DNSSEC makes a difference.

The current Web's CAs system has obvious flaws, and periodic clown shows, but it gets just enough workarounds to maintain the status quo. DNSSEC would like to replace Web's CAs with its own (DANE), but that isn't a fundamental departure from trusting CAs, only a different arrangement of authorities.

aleph_minus_one• 2 years ago

for transport security we use HTTPS (TLS+WebPKI) that doesn't rely on DNSSEC

There exist other application-layer protocols than HTTP.

tialaramex3• 2 years ago

for transport security we use HTTPS (TLS+WebPKI) that doesn't rely on DNSSEC

The Web PKI relies heavily on DNS, and thus without DNSSEC it's vulnerable to spoofing. Of course people like Thomas have worked very hard to get people not to enable DNSSEC, and so a great many domains can be spoofed. Are they? Maybe†. It's hard to tell, after all there would be no sign of a problem, everything looks fine, there's no validation step because you said you didn't want one...

† At a previous employer in 2019 I looked into this. Military intelligence definitely perform attacks on foreign country Internet infrastructure, but at that time they mostly seemed to rely on the fact that users don't use TLS and/or dismiss dialogs saying the certificates didn't match. The UX got better since, so I'd expect today they routinely ask a CA to issue for these unvalidatable domains once they control the packet flow.

troglodynellc• 2 years ago

The core problem with DNSSEC adoption has always been what happens when your ZSK/KSK expires, which it ought to for the same reason SSL certs expire.

Rolling this over in an automated fashion is desirable, as if this just happens to slip your mind, too bad, NXDOMAIN

This is obviously a non-starter for most people; otherwise this would just be automagic like letsencrypt is now.

CDS and CDNSKEY records basically solve this problem, but last I checked only a tiny minority of registrars implement them. Even then, some of them require things like 3-day windows in which the CDS/CDNSKEY must not change before they obey. It's basically a recipe for raising your blood pressure 10mmHg.

So, everyone ignores it for this very good reason. As long as it's essentially installing a landmine in your office chair nobody will touch it.

xz539• 2 years ago

The core problem with DNSSEC adoption has always been what happens when your ZSK/KSK expires, which it ought to for the same reason SSL certs expire.

For most users there's really no reason for a ZSK/KSK split or rolling keys, much the same as there's no need for rolling SSH keys for most users.

tialaramex3• 2 years ago

As far as I can tell this is a BIND bug. There are typically a dozen or so of these each year. The BIND 9 codebase (which is where this bug arose) dates back to when Bill Clinton was President of the US.

Rather than "Ooh, DNSSEC is bad" the correct take is that we need to wean ourselves off ISC BIND. Maybe ISRG can pay somebody to make a less awful DNS server implementation in Rust.

quickthrower23• 2 years ago

The Linux codebase dates back to when George H. W. Bush was president.

IshKebab• 2 years ago

Well... security vulnerabilities in Linux are a fairly common occurrence too!

obscurette• 2 years ago

It MIGHT originated from ISC, but at the moment it's the behavior standard specifies and the fix all DNS software vendors have applied, breaks the standard.

mjevans• 2 years ago

IMO every operation should have a maximum time to execute, and rejecting an update / response for failure to process fast enough SHOULD be valid, even for a an otherwise well formed packet. A protocol must be resilient to valid communications failing for arbitrary reasons including operator error or cosmic bit-flips that corrupt the transmission.

bluejekyll2• 2 years ago

You might be interested in Hickory DNS (recently we rebranded it from Trust-DNS). The ISRG project is sponsoring work on this, announcement here: https://www.memorysafety.org/initiative/dns/

nyrikki2• 2 years ago

The specification is to blame this time, not memory safety.

This is one of those HALT problems that everyone likes to dismiss :)

vdaea• 2 years ago

The BIND 9 codebase (which is where this bug arose) dates back to when Bill Clinton was President of the US.

The fact that it's that old should mean it's extremely robust.

hsbauauvhabzb2• 2 years ago

Age has nothing to do with vulnerability frequency, robustness or presidency. Popularity and investment do.

taftster• 2 years ago

It's a little weird to me that you're dating this back to a timeline associated to a presidency.

I think it just makes me feel old. Like, Bill Clinton's presidency wasn't that long ago. (was it?)

vlovich123• 2 years ago

Depends on whether you think 24 years ago is a long time ago or not.

nyrikki2• 2 years ago

dnsmasq and unbound are impacted to

https://github.com/NLnetLabs/unbound/releases/tag/release-1....

https://lists.thekelleys.org.uk/pipermail/dnsmasq-discuss/20...

As are any other DNSSEC validators that followed the specifications.

Bind9 has its problems but this is not its fault this time.

denkmoon• 2 years ago

Real happy powerdns user sounding off. I've only ever run it for internal dev stuff but it's great.

xz539• 2 years ago

PowerDNS got caught out too: https://blog.powerdns.com/2024/02/13/powerdns-recursor-4-8-6...

tsimionescu4• 2 years ago

The researchers and other DNS server maintainers explicitly say this is a bug in the DNSSEC standard, and technically all patched servers are now standard-incompliant.

zaik2• 2 years ago

As of December 2023, approximately 31 percent of web clients worldwide used DNSSEC-validating DNS resolvers

That doesn't seem right. Ever since enabling DNSSEC validation on my system, YouTube and every other Google product except basic searching is broken for me. The percentage of internet users who enforce DNSSEC must be much much smaller.

bongodongobob• 2 years ago

Yeah, I don't buy that either.

Consulting and working for MSPs over the last 10 years I've probably been exposed to a couple hundred environments and I've never once seen DNSSEC validation used.

neonnomad• 2 years ago

I'm not sure the logic, since you have problems everyone else must have them too? Google DNS / Cloudflare DNS are both DNSSEC validating revolvers, two of the most popular DNS services on the planet. Most US based ISP resolvers are DNSSEC validating (Xfinity's 75.75.75.75 is). Google Chrome ships with DOH going to Google DNS on by default now, Firefox does DOH by default as well.

So I can absolutely see where this would easily be the case, if not more.

You can view a map of DNSSEC validation rates here: https://stats.labs.apnic.net/dnssec

xz539• 2 years ago

Ever since enabling DNSSEC validation on my system, YouTube and every other Google product except basic searching is broken for me.

Enabling DNSSEC wouldn't change resolution of those properties unless you somehow set your system to treat insecure delegations as bogus. Is that what you've done?

api• 2 years ago

Then what’s the point of DNSSEC?

jeroenhd4• 2 years ago

I've had DNSSEC validation enabled for years and Google's services never broke for me. I don't think the problem lies with Google on this one, it's probably a bug in your resolver (or worse, your ISP may be intercepting DNS traffic to Google).

zaik2• 2 years ago

l.google.com has some kind of DNSSEC error [1] but it seems to be required to load youtube.com

[1] https://dnsviz.net/d/l.google.com/dnssec/

stock_toaster• 2 years ago

Some public resolvers like quad9[1] valid dnssec for you. My guess is people using those resolvers is what they are talking about?

That number does still seem high though.

[1] https://quad9.net/support/faq/#dnssec

TacticalCoder2• 2 years ago

by exploiting a 20-plus-year-old design flaw in the DNSSEC specification.

If it's an error in the specification, how can patches already be available without breaking the specs?

Can anyone shed light on this? I'm asking for I'm running unbound which is affected (because it's following the spec IIUC) and yet a patch for unbound is already out.

xz539• 2 years ago

In a nutshell, the vulnerability is stuffing a lot of broken signatures from different keys in a response so the validator wastes a bunch of time retrieving keys and then validating signatures that'll never validate. The fixes just limit the amount of time before validators yield to another task and/or give up. It's a big deal if you run a public resolver but otherwise you can probably fix it at your leisure.

tsimionescu4• 2 years ago

It's similar to how compilers often have limits on compile time constructs that are maybe not explicitly in the standard, but that no one really cares about.

Say, if you design some recursive template in C++ that resolves after a million steps, that might technically be a valid program according to the spec but no compiler will actually accept it (I think the C++ spec actually allows recursion limits, but that's beside the point).

This vulnerability is a bit like not having that limit. So, maybe an RFC to explicitly call out the need for some limit on DNSSEC processing time will be issued, but in practice no one except for attackers should ever come anywhere near the newly imposed non-standard ones.

clwg• 2 years ago

The academics who found this flaw – associated with the German National Research Center for Applied Cybersecurity (ATHENE) in Darmstadt – claimed DNS server software makers briefed about the vulnerability described it as "the worst attack on DNS ever discovered."

This seems a bit hyperbolic, the txid[0] randomization bug found by Kaminsky was objectively worse as it would allow you to poison the cache and in the days before encryption was considered standard.

https://www.computerworld.com/article/2534823/dns-researcher...

kazinator• 2 years ago

How not to use git:

https://github.com/isc-projects/bind9/commits/main

1vuio0pswjnm74• 2 years ago

Here is the paper:

https://www.athene-center.de/fileadmin/content/PDF/Technical...

1vuio0pswjnm74• 2 years ago

This is really a design flaw in DNSSEC itself. It isn't so much a "vulnerability" in any particular software. BIND and Unbound, and subsequently everyone else that follows internet trends, intentionally added DNSSEC support even though DNSSEC was flawed from the outset.

Why was DNSSEC revived from the RFC graveyard^1 and implemented by authors of BIND and Unbound. Cache poisoning. Why does cache poisoning exist. Because people use third party shared caches. Why do people use third party shared caches. That one I will leave for the reader.

There are other, varied options. I use stored DNS data from a variety of sources that I can cross-check against each other. I use iterative resolution instead of recursive. I am happy to discover when websites occasionally change their IP addresses. I want to know. But most websites I visit keep the same addresses for years.

Almost all DNS lookups I do are _not_ on a "phone". The number of IP addresses I need for the "phone" is relatively small.

1. According to djb, DNSSEC was first conceived in 1993. And the charlatans behind it started taking money from the USG sometime in or after 1994.

https://cr.yp.to/talks/2009.10.30/slides.pdf

That is a long time to be awaiting release in some version of BIND. Gee, I wonder why. With this "KeyTrap" demonstration, one might guess that DNSSEC sat around for 15 years because it was never ready for prime time. One might even conclude it still isn't.

tptacek42• 2 years ago

DNSSEC hasn't been adopted at all, and was designed before the first cache poisoning attacks (both the bailiwick-style extra-records attack and the XID prediction attack, which were exploited separately from 1996 through 1997; Kaminsky's attack that combined the two wouldn't come out for another decade). It isn't motivated by any real threat model, but rather by a DoD effort to synergize all the Internet protocols with IPSEC.

1vuio0pswjnm74• 2 years ago

I run DNS servers behind CurveDNS forwarders at home and it works fine. Of course that's not enough to convince anyone except me that it works. There are some nameservers that offer DNSCurve on the internet. This proves that it works.

For example,

   dq a ianix.com 192.5.6.30

   1 ianix.com - regular DNS:
   197 bytes, 1+0+2+2 records, response, noerror
   query: 1 ianix.com
   authority: ianix.com 172800 NS uz5dns2sdrnxskf5lqt46v34cdlfqb9q2lvvmpr95g3l1qh0148sf6.ianix.com
   authority: ianix.com 172800 NS uz5dns1bx64zu3pgn9nm4zfvmh2vy4hpjy7nkjz6qjcu325bg9hzcx.ianix.com
   additional: uz5dns2sdrnxskf5lqt46v34cdlfqb9q2lvvmpr95g3l1qh0148sf6.ianix.com 172800 A 104.207.143.9
   additional: uz5dns1bx64zu3pgn9nm4zfvmh2vy4hpjy7nkjz6qjcu325bg9hzcx.ianix.com 172800 A 104.248.15.206

   dq -s -k dns2sdrnxskf5lqt46v34cdlfqb9q2lvvmpr95g3l1qh0148sf6 ianix.com 104.207.143.9

   1 ianix.com - streamlined DNSCurve:
   229 bytes, 1+2+2+2 records, response, authoritative, noerror
   query: 1 ianix.com
   answer: ianix.com 3600 A 104.248.15.206
   answer: ianix.com 3600 A 104.207.143.9
   authority: ianix.com 259200 NS uz5dns1bx64zu3pgn9nm4zfvmh2vy4hpjy7nkjz6qjcu325bg9hzcx.ianix.com
   authority: ianix.com 259200 NS uz5dns2sdrnxskf5lqt46v34cdlfqb9q2lvvmpr95g3l1qh0148sf6.ianix.com
   additional: uz5dns1bx64zu3pgn9nm4zfvmh2vy4hpjy7nkjz6qjcu325bg9hzcx.ianix.com 259200 A 104.248.15.206
   additional: uz5dns2sdrnxskf5lqt46v34cdlfqb9q2lvvmpr95g3l1qh0148sf6.ianix.com 259200 A 104.207.143.9

The query to 192.5.6.30 is not encrypted because the .com nameservers do not provide a DNSCurve public key prefixed by "uz5" in the subdomain.

The query to 104.248.15.206 is encrypted using DNSCurve. Each packet is encrypted separately. Packets are exchanged via UDP just like regular DNS. DNSCurve predates QUIC.

There is also a free secondary DNS service that will let anyone offer DNSCurve without having to set up CurveDNS forwarders. Assuming they are still in business. I have not tested it. This should put to rest any doubts that DNSCurve actually works. But I know it won't.

https://www.bunnyns.com

tptacek42• 2 years ago

The whole point of CurveDNS (and now DoH) is that it works right away, and doesn't depend on the rest of the Internet cooperating with you. It's a bottom-up design, contrasted to DNSSEC's (failed) top-down model. The only problem with DNSCurve is that it's been effectively superseded by DoH. It's the Betamax of secure DNS protocols. Doesn't matter if it's better.

1vuio0pswjnm74• 2 years ago

DNSCurve is used to encrypt queries to authoritative DNS servers. DoH is only used to encrypt DNS queries to third party DNS caches. Using third party caches can open the door to cache poisoning. Cache poisoning can be and has been used as a "justification" for deploying DNSSEC.

tptacek42• 2 years ago

Right, but (1) most of the value of secure transport for DNSSEC is in the "last mile" between the resolver and the stub resolver on the laptop or whatever, and (2) the same model that secures that hop can secure authoritative lookups for resolvers --- neither protocol is widely deployed for authority queries for recursors, but DoH already has huge deployment numbers for the other use, and seems the more likely bet for how this will play out going forward.

rurban• 2 years ago

Proper link: https://www.presseportal.de/pm/173495/5713546

Just one bad packet can bring down a vulnerable DNS server thanks to DNSSEC

Related Stories