Scaling the Root of the DNS

https://www.potaroo.net/ispcol/2020-09/root.html

by zdw • 6 years ago

98 54 6 years ago

vlovich123• 6 years ago

I wonder if the only change that needs to happen is that applications driving a significant amount of traffic (i.e. Chrome) & DNS owners of other domains need to start bearing the costs here. Then we wouldn't be dealing with cost externalities & tragedy of the commons. On the other hand, introduce revenue streams & you start focusing on optimizing the revenue stream instead of solving the problem.

I wonder if there's a different way to make the costs felt that isn't directly revenue to align incentives. Like slow down or fail queries for zones who's authoritative servers don't implement the features needed to keep scaling under control. Then make sure that there's revenue to the root servers from non-root servers based on load so that costs are tracked & managed.

temp6675• 6 years ago

Just let google run some of the roots - I'm serious. For all the whining about bandwidth and traffic, this issue is so small on the scale of google (which includes youtube) and netflix traffic flows it's hard to even imagine.

Their secondary resolvers run at huge volumes, everything from 8.8.8.8 to the dns for their mail and other web properties.

jeffbee6• 6 years ago

The problem is it does not help to have one or two fast roots. The root server model only really works when the roots each have similar capacity.

JdeBP8• 6 years ago

You've conflated the proxy DNS servers, that do grunt work of query resolution (or forward to other proxies that do), with the content DNS servers. It is not the content DNS servers that are causing the scaling problems, it is the proxy DNS servers, and the DNS clients and applications softwares that drive them.

* http://jdebp.uk./FGA/dns-server-roles.html

The change that needs to happen is the thing that most people nowadays tell one first, that this paper in contrast leaves all of the way to the end: run your own content DNS server(s) for the root, on your own LAN or Intranet. It's fairly simple to do, and it's an idea that has been around for over two decades, now.

* https://news.ycombinator.com/item?id=24277855

* https://news.ycombinator.com/item?id=19526108

* http://cr.yp.to/dnsroot.html (dates back to 2001)

* https://news.ycombinator.com/item?id=24280536 et al.

It takes a long time in the DNS world for such grassroots ideas to percolate to the level that they get circulated by corporate officers. This article is, for example, still propounding that every resolving proxy DNS server works like ISC's BIND, and has a "priming query". This is untrue in reality, and is one of the differences between BIND and some of the softwares (e.g. dnscache and MaraDNS) that came along to replace it a couple of decades ago.

* https://maradns.samiam.org/faq.html#roothints

Similarly the argument that Google should use a subdomain, presented in this article without any counterpoint at all, has long since been countered by pointing out that this makes the malicious actors' lives easier; as conditional forwarding and split-horizon DNS service, which are well-known things, would then make it easier to fool Google Chrome. And there have been other counterarguments made, years ago.

* https://news.ycombinator.com/item?id=3609498 (yes, 2012)

dkdk82832• 6 years ago

I think your terms are incorrect. Proxy queries in your context are referred to as recursive and content is authoritative.

jeffbee6• 6 years ago

It's only 1300 QPS per site. Is just plain old better programming on the table? An ordinary PC should be capable of barfing out NXDOMIAN responses at, conservatively, 100000 per second.

tinus_hn2• 6 years ago

I don’t even know if anyone is even saying there is a problem. All we have is some blog saying traffic has increased.

JdeBP8• 6 years ago

See https://news.ycombinator.com/item?id=24270369 et al..

temp6675• 6 years ago

I'm looking into this a bit more for some basic math.

It looks like QPS is around 1.3M per second? Am I way off? What the bandwidth? I'm getting 1GBps or less which seems pretty low?

If the root can't handle this how does it handle DOS floods even?

Should we just turn some of the root over to some of the bigger players that can actually scale (AWS / GCP / Azure / Netflix etc)? These rates seem so small for this level of complaining about google et al.

temp6675• 6 years ago

Digging a bit more it looks like some of just the public DNS servers are doing WAY more traffic than this. Google was doing 810K qps on 8.8.8.8 alone in 2012 (8 YEARs ago).

Can't they offer to operate some of the root zone then if it's so hard for these other folks to manage it (maybe do something with AWS / Azure etc).

jhalstead• 6 years ago

As of the first graph in this August 2018 blog post [0], it seems like Google's Public DNS was handling about 14-18M QPS (1.2-1.6T per day).

[0] https://security.googleblog.com/2018/08/google-public-dns-tu...

geofft• 6 years ago

Yeah, the conclusion I'm getting is, how do I sign up to be a root? Chrome's NXDOMAIN detection is good for the world and it sounds like handling 50% of root server traffic would not be that hard.

tinus_hn2• 6 years ago

If you install a service like pi-hole, does that query the root servers?

jeffbee6• 6 years ago

No, pi-hole delegates to other resolvers, by default Google 8.8.8.8.

geoffeg• 6 years ago

Not by default, but you can easily install unbound and configure the pi-hole to use it as a recursive DNS resolver.

monkpit• 6 years ago

This could be much more direct. 30 paragraphs in and I don’t really see anything of substance.

chaz6• 6 years ago

How about a system for root zone distribution akin to rpki using bittorrent where updates are signed by a trusted key? Then any resolver could easily be a root server.

akvadrako• 6 years ago

The root zone is already signed and there is a distribution network in place – that's how caching DNS servers work. It's already very scalable.

rasengan2• 6 years ago

Moving the DNS root to blockchain is one of the best use cases for blockchain [1] given the numerous stakeholders (name owners) and type of information required to be in consensus. This instantly serves to distribute DNS root queries, instantly scaling the root.

[1] https://handshake.org/

CharlesW• 6 years ago

If one of the best use cases for blockchain is "data about name owners" and "DNS information", the future of blockchain is bleaker than I'd imagined.

Handshake isn't a super-persuasive example to cite, since it's backed by a minor altcoin that's primary use case (according to CoinDesk) is crypto day-trading.[1][2]

[1] "Update (May 28, 20:11 UTC): The headline and body of this article have been edited to clarify there were reportedly $10 million worth of HNS trades on Namebase since the token launched, but* only a fraction of those users bought domain names." https://www.coindesk.com/handshake-domains-bring-in-10m-as-r... [2] https://davidgerard.co.uk/blockchain/2020/05/31/news-not-cra...

rasengan2• 6 years ago

That article likely didn't review the activity on chain [1].

[1] https://dns.live/

throw0101a4• 6 years ago

Chrome (and derivatives) need to start behaving better:

The motivation for the Google’s Chrome browser to do this is pretty obvious. Many ISPs perform NXDOMAIN substitution in their DNS resolvers and replace the “no such domain” response with a synthetic pointer to their own search page, which allows them to monetize all those No such domain” user typos. From Google's perspective this NXDOMAIN substitution is not well regarded. Search is a major path to advertisement placement and advertising is Google's core revenue. So how can Google's Chrome browser platform detect network environments where NXDOMAIN substitution is happening? Simple. Test the DNS with its own nonce random string queries to see if NXDOMAIN substitution is taking place.
This supposedly innocuous probe of the DNS should’ve been pretty harmless. But there is a lot of Chrome out there these days. Some two thirds of all browser use in today’s Internet is reported to use the Chrome browser, and the number rises when you include products such as Edge that are based on the Chrome engine. Consequently, it is perhaps unsurprising to learn that according to this report these probes are now taking up some 50% of the total root server traffic (Figure 3).

I understand that hijacking NXDOMAIN is bad, but Google is now 'punishing' the root DNS operators with extra traffic when the actual perps are the bad-actor ISPs. A bit of collateral damage there.

And would it be so bad for Google/Chrome to simply let the NXDOMAIN hijacking slide? They seem to be externalizing costs to help their business model.

remus• 6 years ago

And would it be so bad for Google/Chrome to simply let the NXDOMAIN hijacking slide?

Yes, it would. It's a workaround to a problem created by shitty ISPs, go after the ISPs if you want to solve the problem.

throw0101a4• 6 years ago

Great. And what is your idea for to "go after the ISPs"? What does 'going after' them entail?

And as someone with a non-shitty ISP, why should I be part of the problem of excess traffic to the root servers?

JdeBP8• 6 years ago

You need to know a bit of history, namely that one of the earliest "bad actors" here was Verisign, the operator of the com. and net. registry, and of the "A" and "J" ICANN root content servers.

* http://jdebp.uk./FGA/verisign-internet-coup.html

schoolornot2• 6 years ago

What's the rationale for not hardcoding Google DNS into Chrome the way Firefox has done with Cloudflare for DoH?

throw0101a4• 6 years ago

Split-horizon DNS in corporate networks for one: how would Chrome look up internal-only records?

(Which is one of the reasons that many sysadmins don't like Firefox's DoH, though not the most important one.)

throwawaysea• 6 years ago

My knowledge is limited here but at some level there is a limit to the amount of traffic a single endpoint can handle right? For example if something is distributing queries in a round robin manner, isn’t there a limit on how much load that can ultimately handle and distribute to other computers? What is that limit and what constrains it?

wiml• 6 years ago

If only there were some technology that could verify whether a DNS query/response had been spoofed, without interfering with caching and proxying, and without the privacy and centralization-of-control problems of DoH. If only such a technology had already been widely deployed in the dns root and many caches and resolvers.

NKosmatos2• 6 years ago

Out of curiosity, how big (in GBs/TBs) are the root files? If I setup a local DNS server (Raspberry or something more powerful) could I duplicate or cache the root file somehow? IMHO the various ISPs should contribute to the cost and technical needs of the root servers or even better utilize blockchain, torrent or other distributed network concepts.

vesinisa2• 6 years ago

The root zone has grown considerably thanks to a number of new gTLDs, but currently sits at around 2 MB:

https://www.iana.org/domains/root/files

That's in human-readable ASCII. A wire format could probably shave a chunck off that, but distributing deltas instead of re-downloading the whole file each time a change is made would further reduce the required data transfer amounts to mere bytes per day on average.

NKosmatos2• 6 years ago

Thanks for your reply, I was thinking about the actual files containing the DNS entries and not the root file which contains all of the DNS servers. As I understand it the root file is needed in case you want to bootstrap your own DNS server so as to be able to find all the other authoritative servers. What about the actual DNS database files?

1vuio0pswjnm7• 6 years ago

   ftp -4o/dev/fd/1 ftp://192.0.32.9/domain/root.zone|dd >/dev/null

   2165210 bytes (2.2 MB, 2.1 MiB)

Anyone can serve their own root.zone. I have done so for many years. The web might be "big" but the portion of it you as a personal user are accessing is comparatively small. If you tally up all the domains you are currently visiting (excluding nonsense like ad servers), chances are, a zone file that contains the RRs for every domain you will ever need to visit in your entire lifetime would fit on a small-sized USB stick from 10 years ago. You can effectively avoid recursion. 100% authoritative DNS, all served from your computer.

Today's USB sticks can accomodate the largest zones like .com several times over. That is more web than you could possibly surf in your lifetime.

Scaling the Root of the DNS

Related Stories