Show HN: File distribution over DNS: (ab)using DNS as a CDN

https://eighty-twenty.org/2023/07/31/dns-as-a-cdn

by tonyg • 3 years ago

72 34 3 years ago

TL;DR. It works, more or less, so long as your resolver properly upgrades to DNS-over-TCP when it gets a truncated UDP response.

Coincidence that MUSL just added support to DNS-over-TCP fallback? https://news.ycombinator.com/item?id=36933028

tonyg5• 3 years ago

Heh, actually yes; the universe trying to tell me something, maybe?

tonyg5• 3 years ago

(Has anyone tried the little demo? ... Does it work?)

jesprenj3• 3 years ago

There's also iodine, a C program that tunnels IPv4 packets over DNS. Useful for bypassing captive portals on wifi, since DNS usually isn't restricted.

https://github.com/yarrick/iodine

Regarding cloudflare DNS over HTTPS: It could be that it tries to server data encoded as JSON, which is impossible in JSON. Some control characters and bytes 128-255 cannot be represented as JSON strings.

OJFord2• 3 years ago

IME they block DNS other than their own (not even rewrite, just outright block). Not through experience trying to use iodine, but because I frequently have to drop my explicit DNS server in order to be able to reach the captive portal and connect legitimately.

smashed• 3 years ago

Of course. A captive portal can also be more sneaky and mangle the packets to redirect them to their own DNS service, since it's not encrypted.

But that's the beauty of iodine. It will still work because if the captive portal's name servers actually fully resolve requests, it will contact your upstream iodine controlled name servers and forward the response as-is, because that's just how DNS works.

Of course it's also fairly easy to detect/block since your DNS usage will be completely abnormal.

jesprenj3• 3 years ago

Yeah, like a sibling comment replied, the idea of iodine is that you don't use your own resolver but the one that the captive portal provided.

You need a domain name, xxx.example, then you can uplink data yyyyyy through the tunnel by querying yyyyyy.xxx.example and iodined, authoritative for xxx.example, will receive this query by the captive portal's resolver. To downlink data, you read responses (txt, cname or null records) that iodined will send for your queries.

This probably fills up the cache of the captive portal's resolver (:

derN3rd• 3 years ago

Wouldn't it be simpler and easier for clients to implement when they would use the TXT to store a magnet link and host the file via webtorrents?

Maybe such a solution already exists, but I couldn't find it

imoverclocked• 3 years ago

It's not far from OpenAFS which uses SRV records to point to fileservers. It's not magnet/torrent but it's certainly DNS discovery for data on a different protocol.

tonyg5• 3 years ago

Well, I'm not sure adding a webtorrent implementation counts as simpler for clients than just TXT record retrieval.

whalesalad• 3 years ago

Nope. How could you get any easier than plain DNS? That would require a torrent client and bittorrent protocol. The beauty of this hack is that it exists on top of the ubiquitous DNS system.

dannyobrien• 3 years ago

I think the obvious match here is how IPFS can use a dnslink TXT file to point to the content-addressable top of an IPFS-distributed static web https://docs.ipfs.tech/concepts/dnslink/ . This is how a lot of dweb/web3 websites maintain and advertise a non-http mirror. It works out of the box for sites served using https://distributed.press/ (non-commercial, open source) or https://fleek.co/ (hosted).

If you use Brave, or have the IPFS browser extension, you can access sites like https://ffdweb.org/ as an option, or preferentially.

arjvik• 3 years ago

NSCDN? You missed the chance to name it CDNS!

victorbjorklund• 3 years ago

DNS is wild. I really need to dig down and better understand it.

kalupa• 3 years ago

I see what you did there ...

rpastuszak3• 3 years ago

wait, who is wild?

woleium• 3 years ago

Don't do this. DNS works and continues to work because we don't abuse it.

RockRobotRock2• 3 years ago

Pushing limits is how we get innovative technology. It's interesting, if nothing else.

cookiengineer• 3 years ago

Tell that to hackers doing DNS exfil since 10+ years.

Modern ransomware is even able to abuse/imitate apple's airdrop and other DNS-SD services because they're let through by most configurations.

1vuio0pswjnm7• 3 years ago

"Hmm. Promising. Unfortunately, it doesn't seem to like the TXT records having binary data in them (bug? certainly TXT records are allowed to hold binary data, see here and then here), so it might not Just Work."

Aside: IMHO, dnstxt from djbdns is easier for requesting TXT records than dig; it's a much smaller, simpler program.

tinydns from djbdns can store any data in TXT records, i.e., arbitratry bytes specified by octal. Perhaps other authoritative servers can also do this today. At the time djbdns was released AFAIK it was the only one.

"TXT (``text'') record for fqdn. tinydns-data creates a TXT record for fqdn containing the string s. You may use octal \nnn codes to include arbitrary bytes inside s; for example, \072 is a colon."

https://cr.yp.to/djbdns/tinydns-data.html

Thus one could, e.g., store mini-web pages in TXT records. I experimented with this about 15 years ago.

m30472• 3 years ago

A lot of our current tech stacks are topheavy. I'm all for re-examining foundational internet technologies and objectively examining how stuff can be pushed down the stack as it were; I'm arguably one of the people destroying DNS if that's your mindset, seems like a good key-value store and "data diode" to me.

I think email (SMTP) still hasn't been fully exploited as a job scheduling tool (events... mini batches... SMTP).

Bearing in mind, I don't expect people to burden the public infrastructure with these kinds of activities. I think if you're going to use the DNS for telemetry you need to scale your deployment to handle at least hundreds of requests a second; this isn't hard with DNS, it's just not the mindset of DNS operators intending to use the DNS as it always has been used. There are also some blatantly political considerations which need to be swept away to make it truly seamless.

The gnashing of teeth is obviously suspect when you consider the trajectory of HTTP: from web pages to XHR, SOAP and DoH. However questionable you might find the architectural choices, HTTP continues to work in spite of the abuse. Likewise, the demise of DNS is unlikely.

dvasdekis• 3 years ago

Have a think then about modern encryption and consensus on top of Usenet, which is the SMTP-compatible message queue of your dreams (in terms of available capacity)

m30472• 3 years ago

I have thought about it, and some "modern" encryption but maybe not what you're thinking of.

rpastuszak3• 3 years ago

I like that, I use Twitter/X as my CDN: https://sonnet.io/projects#:~:text=Laconic!%20(a%20Twitter%2...

quickthrower2• 3 years ago

CDN? or REST DB?

Potato potato I guess

rpastuszak3• 3 years ago

Somewhere between S3 and Glacier depending how you look at it.

Potato potato I guess

I'd go with potato. I actually own https://potato.horse and now I'm starting to think that I should point cdn.potato.horse to that twitter thread!

unixfox• 3 years ago

Somewhat related article: https://blog.benjojo.co.uk/post/dns-filesystem-true-cloud-st... (DNSFS. Store your files in others DNS resolver caches - 2018)

tonyg5• 3 years ago

Very interesting and relevant! Thank you!

RockRobotRock2• 3 years ago

This is definitely a harder drive.

askiiart• 3 years ago

Context: https://youtu.be/JcJSW7Rprio

tbiehn• 3 years ago

Yet another DNS file delivery thing - wrote a few years ago to get files into restrictive networks and results out: https://dualuse.io/blog/dnspump/

tonyg5• 3 years ago

Cool, thanks for sharing!

Show HN: File distribution over DNS: (ab)using DNS as a CDN

Related Stories