DNS Flag Day on February 1, 2019: check your domains

by el_duderino • 8 years ago

375 93 8 years ago

jaachan2• 8 years ago

I can't really find a simple explanation of what this means. The warnings in the EDNS compliance tester aren't really helpful either. Is there a simple explanation somewhere?

LeonM2• 8 years ago

"Starting February 1st, 2019 there will be no attempt to disable EDNS as reaction to a DNS query timeout. This effectivelly means that all DNS servers which do not respond at all to EDNS queries are going to be treated as dead."

So it basically means that authoritative DNS servers are now required to support the EDNS protocol. If not, it is no longer guaranteed the domain will resolve on DNS resolvers.

This is a performance improvement, because the EDNS fallback method requires a timeout.

Edit: My answer isn't correct, see the comments below.

Twirrim• 8 years ago

What is EDNS, and why would people want it?

jaachan2• 8 years ago

Testing my DNS server gets me:

    edns=ok edns1=noerror,badversion,soa
    ednsopt=ok edns1opt=noerror,badversion,soa

Wikipedia doesn't mention anything about EDNS versions, so I'm not sure what to make of it.

JdeBP4• 8 years ago

No. It means that content DNS servers (and their concomitant network infrastructure) must either support EDNS properly or ignore it properly. The halfway house of having clients fall back to re-trying without EDNS, because some bad servers failed to send replies (or the network infrastructure that they communicated over failed to send on those replies) in response to EDNS queries, is going away.

And about time, too.

* http://jdebp.eu./FGA/dns-edns0-and-firewalls.html

zAy0LfpBZLC8mAC• 8 years ago

No, authoritative servers are simply required to follow standards, EDNS support is not required, none of this is about any sort of forced migration, it is only about discontinuing support for broken software after 20 years of compatibility hacks.

The EDNS fallback method does not require a timeout. A standards-compliant DNS server that does not support EDNS should ignore EDNS records in the request and simply send an old-style response that any EDNS capable resolver is still required to handle correctly. The problem is only with servers that simply don't respond to requests containing EDNS records at all, even though they are required to by the (old, pre-EDNS) standard.

bcaa7f3a8bbc4• 8 years ago

Just my two cents.

To most people, it's not the "domains" that should be checked, but DNS servers and DNS resolvers, both the authoritative and recursive type. If you are using a major DNS provider for your domain, no action is needed, but just to be sure, use the test tool on the webpage to see if your provider has broken EDNS, and do check your local recursive server.

Classic DNS messages carried by UDP were restricted to 512 bytes, EDNS boosted this restriction and also introduced some flags, and it has been enabled by major DNS servers since 1999. But in practice, many deployments on the authoritative servers are broken, they signal EDNS support, but EDNS replies are silently dropped, due to broken DNS servers, misconfigured router, broken NAT, broken ISP installations, or broken firewalls or other middleboxes.

Previously, various DNS resolvers contained a workaround that disables EDNS as reaction if a DNS query timeout is detected. Now the workaround will be removed. If a DNS resolvers has EDNS but it's broken, it will be marked as a dead server.

nottorp• 8 years ago

Yeah, for a compliance tester it's surprisingly unhelpful. What do I need to do if my DNS is "Slow" ?

Thanks HN commenters for doing the site's job :)

rconti2• 8 years ago

Yes, this page provides virtually no detail either before or after getting a SLOW or SOME PROBLEMS result. I spent a lot of time in the weeds of EDNS a couple years back and I tested that company's domain, and have frankly no idea what the results mean, what the change is, etc. Seems like a source article trying to encourage change could have been a little helpful beyond "upgrade yo' shit" without explaining what it means.

If you're running a DNS server and have problems, it's probably not a simple matter of blowing out the latest version of your software. You probably have a rat's nest of thousands of entries added over a decade or more, different firewall policies in front of different authoritative servers, caches, etc.

breischl• 8 years ago

This is a bit trivial, but why is it called "Flag Day"? Because they're flipping some configuration flag to change the behavior?

bcaa7f3a8bbc4• 8 years ago

Flag Day means a software change that is not backward-compatible, especially in a network system where upgrade is scheduled early and everybody must deploy upgrade at (before) the date, or the system will stop working. It came into use when a change was made to the definition of the ASCII character set during the development of Multics. The change was scheduled for Flag Day (a U.S. holiday, https://en.wikipedia.org/wiki/Flag_Day_(United_States) ), June 14, 1966.

See Jargon File.

http://www.catb.org/~esr/jargon/html/F/flag-day.html

arayh• 8 years ago

Also see Wikipedia: https://en.wikipedia.org/wiki/Flag_day_(computing)

CedarHill• 8 years ago

Can some ELI5 on this? Not entirely sure what's actually happening.

pas2• 8 years ago

EDNS is basically DNS2, and it has better support for handling larger messages (needed for DNSSEC and such), and EDNS itself is about signaling what features you support.

https://www.dns-oarc.net/node/93

JdeBP4• 8 years ago

Read http://jdebp.eu./FGA/dns-edns0-and-firewalls.html for an explanation.

What is happening is that all of the stuff that I have been explaining in the "Local fix" section for a decade and a half, alongside the practice that clients had of timing out and then retrying the transaction without the extensions, is now being declared moot. If DNS lookup does not work because of this problem, it is being declared entirely the server-end's problem. Clients are no longer going to be expected to put such local fixes in place, or have timeout and retry bodges.

kstrauser• 8 years ago

As someone who's been required to maintain bug-compatible software before, this fills me with great joy. Congratulations on getting to deprecate the brokenness!

patrickg_zill• 8 years ago

Hasn't dns been working for the past 25 years?

kps• 8 years ago

Close, 35 years.

https://tools.ietf.org/html/rfc882

https://tools.ietf.org/html/rfc883

detaro3• 8 years ago

More than that: DNS servers made special exceptions for servers that do not implement DNS properly and ignore some requests entirely. They're now stopping that: servers either have to support a 20-year old extension spec to DNS, or follow the original, 30+ year old specs and answer the query while ignoring the extension.

pasta• 8 years ago

Wow, EDNS is a DNS extension mechanism first published in 1999.

So I can imagine after 20 year it is time to move on.

When I first read it I thought it was some kind of 'new' tech.

tialaramex• 8 years ago

More over, this isn't about "you have to implement EDNS" (you should, but you don't have to). This is about "Either implement EDNS or when asked for EDNS, just say you don't offer that".

This isn't so much "You must offer a vegetarian option" as "Don't offer a vegetarian option but then when anybody orders it you force a steak down their throat while screaming 'Choke on it you hippie!'".

kawsper• 8 years ago

Testing my domains on Route53 gives me this:

Minor problems detected!
This domain is going to work after the 2019 DNS flag day BUT it does not support the latest DNS standards. As a consequence this domain cannot support the latest security features and might be an easier target for network attackers than necessary, and might face other issues later on. We recommend your domain administrator to fix issues listed in the following techical report: redacted.

bcaa7f3a8bbc4• 8 years ago

It means the authoritative servers of your domain simply does not support EDNS (a 20-year-old protocol...), but it doesn't falsely advertising EDNS support, which is what the Flag Day's targeting, so your situation is good.

colmmacc3• 8 years ago

Route53 absolutely does support EDNS0 and large answers without needing to truncate.

The minor error reported is that Route 53 does not respond with a "BADVERS" error in response to a query reporting EDNS version 1.

There is no EDNS version 1 standardized, and the specs say to respond with an error when you see a version you don't support. Per the specs Route 53 is "in the wrong". The thinking from the spec authors is that this kind of behavior will make it easier for resolvers to negotiate future EDNS versions and figure out what works. I'm skeptical that this will ever actually work in practice: most protocols tend to have backwards-compatible version upgrades that don't require round-trips.

The thinking on the Route 53 side, or at least my thinking - I wrote Route 53's EDNS support - is that we see incorrect versions occasionally on the wire, from weird clients where people have clearly made some kind of mistake, and that it is better to fail gracefully and give them /an/ answer than an error that could lead to outages. I might be completely wrong about that, but that's the thinking. We tend to always heavily lean in the direction that is likely to increase availability and avoid any potential for outages.

priansh• 8 years ago

Same for me. All good for 1and1 and GoDaddy, but my AWS domains all have "minor problems detected!"

I assume (hopefully) that Route 53 is going to upgrade eventually.

brooksgarrett• 8 years ago

Here is an AWS developer discussion that details their response:

https://forums.aws.amazon.com/message.jspa?messageID=851817#...

Quote for those without access: There is a known bug with Route 53 DNS servers in which we are not RFC compliant in how we handle a specific kind of invalid query. Namely, when Route 53 gets a query with an unknown EDNS version, Route 53 treats the query as a non-EDNS query instead of responding with BADVERS as ednscomp expects.

We expect to have this bug fixed within a year or two. The good news is that this bug is not impacting, so you'll be ok even if we're slow to fix this. The "dnsflagday" news means that servers that don't support EDNS will be treated as unavailable to resolvers. But Route 53 generally supports EDNS0, valid queries will continue to work regardless.

wodenokoto• 8 years ago

You might want to test your domain more than once. First I was told my domain was in trouble, but after retesting I was told everything is a-ok ...

dttocs• 8 years ago

This likely means that your DNS server IP is actually a load-balanced pool of servers, with some running compliant software and others not. If you see any errors when running the test repeatedly, I’d check with your DNS provider, and share the test results.

tzs• 8 years ago

If you have N domains, all served from the same DNS server, you only need to test one of them with their tester, right? This is about whether or not the server handles the EDNS protocol, not about anything to do with the content of any particular domain records, so it either works for them all or for none of them?

detaro3• 8 years ago

yes

Kpourdeilami• 8 years ago

Testing my domains on CloudFlare gives me:

Serious problem detected!
This domain will face issues after the 2019 DNS flag day. It will work in practice, BUT clients will experience delays when accessing this domain. We recommend you request a fix from your domain administrator!

Weird since they are listed as one of the supporters

bo0tzz• 8 years ago

I had the same result, but after trying again I got an OK.

bcaa7f3a8bbc4• 8 years ago

I think the test works by testing querying an authoritative server with EDNS on, and see if the request times out. Naturally, there will be false positives due to random packet loss on the Internet.

ogud• 8 years ago

Test again, CF systems are fully ENDS0 compliant https://ednscomp.isc.org/ednscomp/9c45882a1c

Tepix• 8 years ago

How does tinydns deal with it? Does it need patches? Are they available? I don't see it mentioned.

nightbrawler• 8 years ago

Looks like you can add support via the patches mentioned here: https://www.tinydnssec.org/install.html

The home page mentions that EDNS queries will be honored even for non-DNSSEC queries.

JdeBP4• 8 years ago

tinydns speaks the unextended protocol, but properly handles clients that try to speak EDNS0 to it.

It does not exhibit any of the faulty behaviours that are common to bad EDNS0 implementations. It does not blindly echo various parts of the query back in the response, for example. Indeed, the ISC test actually complains that tinydns does not blindly echo an OPT record back. (-:

But you cannot speak DNS/UDP to it with datagrams over 0.5KiB. Patches are needed to make it do anything but wholly ignore EDNS (of any version) and treat it as though it were the unextended protocol. But that is not what this flag day is about.

Indeed, given that tinydns does not send DNS/UDP responses greater than 0.5KiB, it does not trigger any EDNS0 problems in the network infrastructure that a server is using, either.

Ironically, one of the people who tried to rewrite djbdns to have EDNS0 support discovered exactly the problem that this flag day is about: The bad content DNS servers for outlook.com in 2017-2018 gave bad responses to clients that attempted to speak EDNS0.

* http://www.nemostar.org/djbdns/changes.html

rconti2• 8 years ago

One pernicious problem I had with tinydns and EDNS support was in an environment where one firewall, for whatever reason, did not permit tcp/53. So DNS replies worked, EDNS0 replies worked, but not from certain clients that insisted on retrying the query over TCP.

Needless to say, after tons of time getting into the weeds with EDNS, and STILL not understanding why certain clients weren't working, that was a fun one to solve.

Panino• 8 years ago

Tinydns doesn't need patches to continue working after Feb. 1, 2019. The article website is about authoritative servers that give broken EDNS answers. You either want to correctly support EDNS, or send normal pre-EDNS answers. Both work, and tinydns does the latter.

I don't know of any latest-version DNS software that breaks in this case. AFAIK this is exclusively a middleware problem, as is quite often the case.

vertline32• 8 years ago

My sites said all okay, I guess, what is this looking for exactly?

zzzcpan• 8 years ago

https://ednscomp.isc.org/compliance/summary.html

vertline32• 8 years ago

Thanks sir :)

uniformlyrandom2• 8 years ago

So, this is just a front-end for https://ednscomp.isc.org/ednscomp, which is a front-end for https://gitlab.isc.org/isc-projects/DNS-Compliance-Testing.

I kinda get the need for a web frontend for a C program (portability is an issue), I miss the point of the second front-end. Why?

codetrotter• 8 years ago

Because the OP link is spreading awareness. Your link states nothing about what it does and probably the output of the one you linked is much less understandable to the population at large as well.

uniformlyrandom2• 8 years ago

Well, I guess 'DNS flag day' is a catchy name, and the text is intentionally FUD-y, so it has better chance of viral distribution. I just like the concise summary on ISC page better, needs less scrolling.

Kye• 8 years ago

Can I safely assume my Netlify-hosted domains will work?

LeonM2• 8 years ago

Depends, if you have hosted the DNS at netlify, then you can assume it is.

If you only run hosting at netlify (thus, having your DNS CNAME to Netlify) you can still have problems.

Use the checker on the website to know for sure.

sandGorgon• 8 years ago

which is the cheapest DNS service that allows one to buy "Registry Lock" (not the simpler registrar lock). I'm talking out-of-band confirmation of any DNS changes ?

The big ones like Mark Monitor, etc are super expensive. I thought Cloudflare Business DNS would be cheaper, but they start at 1000-2000$ per year.

Any recommendations ? I have heard of Safenames..but not many others.

megous• 8 years ago

It depends on the TLD. For example for my national domain, I can just send a request form with my signature to the national TLD registry operator, and that's it. My domain becomes untransferable, and I also have an option to disable change of NS set.

It's free, except for the official signature verification, that can be done at the post office for $1.5.

aquabeagle• 8 years ago

You could open an OpenSRS reseller account and move your domains there, and effectively be your own registrar.

peterwwillis• 8 years ago

Woohoo!!!

The industry's complacency in getting broad EDNS support has been a huge embarrassment imnsho. This is proof that we don't need to put up with "but the middle boxes, argh!!?" excuses if the industry works together to force people to be compatible with standards.

Thanks to all those involved with making this project a reality. It really helps everyone on the internet, whether they know it or not.

aidenn0• 8 years ago

Is there a simple command-line option I can run to check if the caches between me and the internet are ready for this?

oriettaxx• 8 years ago

https://www.isc.org/blogs/dns-flag-day/

rthomas6• 8 years ago

Netlify domains get all green

LinuxBender• 8 years ago

Minor issues for ycombinator.com [1] They use AWS Route53 so could be false positive. I see this for anyone using Route53.

[1] https://ednscomp.isc.org/ednscomp/39674a3069

etbusch• 8 years ago

Network Solutions is failing the test currently, anyone have info on when they might become compliant?

tcombinator• 7 years ago

I've got 2 clients using them and wish I could say this info surprised me, but they are a terrible registrar & DNS provider. I've asked both of my clients to reach out to them about the issue but don't have any information past that.

heyjudy• 8 years ago

att.com is broken. Surprise, surprise.

slackwill• 8 years ago

ycombinator.com has some issues ;)

jcoffland• 8 years ago

Tested my Debian based server running bind, all good. Thanks Debian!

eikenberry2• 8 years ago

Presumably they are not going to remove the workarounds until the software has actually had a chance to disseminate? For example the resolvers they list are all new releases and aren't in any current Linux distro. Which means they are saying that in a couple weeks all Linux resolvers could stop resolving some domains unless they adopt an unsupported release of the resolver?

They don't list it, but presumably the same applies to the Authoritative servers. I run a pretty vanilla NSD install and it fails several tests. NLNETLABS (the authors/maintainers of nsd/unbound) is listed as a supporter, but they don't mention anything about this on their site nor does their documentation mention anything about settings to control these features.

If this will actually have the consequences they describe it was terribly mismanaged.

wjoe• 8 years ago

I am a bit confused about the versions they say are safe. I'm running Bind 9.11 and their tester says there are no issues, and it supports all the necessary EDNS behaviour.

I think the point is that those versions mentioned will remove support for the bad behaviour, but it's not clear what you do need to do to prepare. I guess the only steps you can take are to run the tests and update to the latest version available.

eikenberry2• 8 years ago

Just to clear the record for NSD users... I misread the results and my NSD setup (4.1.0) does pass the test. It was my secondary at buddyns.com that is having issues. I'll report the issue to them, but I'm still concerned about the supported recursive resolvers not being compliant.

wila• 8 years ago

The default powerdns server in debian stable also fails. This is a reasonable common setup for DNS servers. It is a bit weird to have a server running current/stable that does not pass the test.

Apparently the fix is to use the external repository from powerdns.

detaro3• 8 years ago

The resolvers listed are the ones that implement the new behaviour of not including the workarounds. They are not the first versions to support EDNS.

skywhopper• 8 years ago

This page eventually gets around to the point, but the intro paragraphs don't tell me what's actually happening, and are written in an extremely confusing way. I almost just closed the tab since it's not even clear that any explanation is coming.

"This change affects only sites which operate software which is not following published standards. Are you affected?"

Affected _in what way_? Instead of linking to the Wikipedia page about what DNS is (if your visitor doesn't know that, then the rest of the page is pointless), there needs to be a sentence or three up top telling me what is _actually happening_ and what the _actual impact_ could potentially be, and for that matter, which standards you are even talking about. Then, tell me what the forms you are asking me to type my domain names into are going to do.

As it is, this page is just going to leave a lot of people confused.

jethro_tell• 8 years ago

I almost just closed the tab

I did, then I came here to see if you stuck it out. Thanks!

bigbugbag• 8 years ago

this page explains things better: https://www.isc.org/blogs/dns-flag-day/

A number of DNS software and service providers have announced that we will all cease implementing DNS resolver workarounds to accommodate DNS authoritative systems that don’t follow the EDNS protocol.

Domains served by DNS servers that are not compliant with the standard will not function reliably after February 1, 2019, and may become unavailable.

If your company’s DNS zones are served by non-compliant servers, your online presence will slowly degrade or disappear as ISPs and other organizations update their resolvers. When you update your own internal DNS resolvers to versions that don’t implement workarounds, some sites and email servers may become unreachable.

Why make this change now?

Extension Mechanisms for DNS were specified in 1999, with a minor update in 2013, establishing the ‘rules of the road’ for responding to queries with EDNS options or flags. Despite this, some implementations continue to violate the rules. DNS software developers have tried to solve the problems with the interoperability of the DNS protocol and especially its EDNS extension (RFC 6891 standard) by various workarounds for non-standard behaviors. This is not unlike the way a driver with the right-of-way might hesitate at an intersection before proceeding if there were another driver in the intersection behaving erratically. These workarounds excessively complicate DNS software and are now also negatively impacting the DNS as a whole.

The most obvious problems caused by these workarounds are slower responses to DNS queries and the difficulty of deploying new DNS protocol features. Some of these new features (e.g. DNS Cookies) would help reduce DDoS attacks based on DNS protocol abuse.

runjake• 8 years ago

This is going to flop.

I am very familiar with DNS. I run the tests and get minor failures. The test results barely make any sense and don't really offer good pointers on how to resolve them.

The documentation is obtuse in typical ISC fashion.

I could probably figure it out with more effort. But the average DNS Joe won't, they'll ignore this and move on.

tony_g• 8 years ago

I have conflicting thoughts on this. I'm having a problem with the credibility of this whole thing. Very few people know about it. Twitter @dnsflagday has 94 followers, telling me it's just an individual account with no public validation yet. The ISC.org site also doesn't seem to be authoritative.

I understand the technical problem but I'm not convinced this is as big a deal as it seems. There are some big names associated with the flag day event, lending credibility. But there are also big names like Slack.com, Twitter.com, GitHub.com, BitBucket.com, and web hosting companies, that all fail the EDNS/dig test.

So - the stated threat is that sites will go dark on 1 Feb, but do we really expect that to happen? Might these companies simply be threatening to reject invalid EDNS responses? Will people blame Them for rejecting queries rather than the services being resolved and their DNS?

I can't decide if this is something to scream loudly about, or to just let it happen. I'm not asking for opinions on that. I'm asking for a credible source of information to confirm what is truly expected.

Thanks.

roryokane• 8 years ago

The site doesn’t claim that slack.com, twitter.com, etc. will go down when the flag is flipped. Just try entering those domains into the testing widget within the “Domain owners” section, and you can see that their status is not so dire:

Minor problems detected!
This domain is going to work after the 2019 DNS flag day BUT it does not support the latest DNS standards. As a consequence this domain cannot support the latest security features and might be an easier target for network attackers than necessary, and might face other issues later on. We recommend your domain administrator to fix issues listed in the following technical report [link to report]

As other comments have said, the new rule does not require that all domains support EDNS, merely that all domains respond with “EDNS not supported” when applicable instead of pretending that they support it. Those sites you linked follow that rule.

DNS Flag Day on February 1, 2019: check your domains

Related Stories