Theseus DHT Protocol (2018)

https://wootfish.github.io/theseus.dht/

205 39 3 years ago

wootfish.github.io

remram2• 3 years ago

Does it allow someone with knowledge of a key to find the IP addresses of other people interested in that key? Because if yes, the other security guarantees are not all that interesting.

The fact that Tor is mentioned would suggest that you have to add that to be secure.

eis• 3 years ago

I just had a brief read of the protocol but my understanding is yes, there is nothing like onion routing or similar that could disguise to a peer that is serving the data that the requester is indeed the one who is interested in it.

vesinisa• 3 years ago

Maybe this is also why they underline in the preamble that it is suitable for running over Tor.

remram2• 3 years ago

If I can still get fined or disconnected by my ISP for what I do on the network, the whole "indistinguishable from random noise" and "forward secrecy" really mean nothing as an end user.

sohle5• 3 years ago

Hi, I wrote this post, and yes, you're right that IPs would be disclosed, because it is necessary to learn a remote peer's IP before you can make a TCP connection to them. Whether or not this is "secure" depends on who you're trying to be secure against. If you're sharing Linux ISOs, you really don't care who sees you do it; if you're sharing pirated movies, you just care that Disney doesn't find out it was you, so you can use a VPN; if you're sharing government secrets, you might have a more rigorous threat model.

For users in extreme circumstances, Tor gets them a lot, but it also adds a lot of overhead; you'd want to avoid it if you don't need it, not just for your own sake but also for the sake of Tor's finite bandwidth. There is a lot to be said for having a feature like Tor be supported-but-optional, opt-in, and interoperable with users who are not on Tor.

I did consider the idea of allowing Tor hidden service addresses as an alternative identifier to IPs, though that idea comes with its own compromises: you'd only be able to talk to other Tor peers, for instance, or, alternately, every peer would have to be on Tor.

The latter case is obviously a non-option. The former case would kind of work, but the Tor-hidden-service peers' effective network size would be much smaller (since they can only talk to Tor peers) and thus easier to compromise. This is obviously bad - users in need of greater security would end up in a less secure version of the network - and if this subsection gets owned, that could have a second-order impact on the network's overall resilience.

Retr0id2• 3 years ago

I learned about Elligator quite recently and was shocked by how rarely it's actually been deployed (or even implemented), very happy to see it being used here.

Release date: 4/20/2018
Revision date: 10/8/2018

Would be nice to get a (2018) in the title.

bfung• 3 years ago

That’s also only the specification release/revision dates.

Digging into the GitHub repo: “The implementation is not yet complete.”

https://github.com/wootfish/theseus.dht

Retr0id2• 3 years ago

I found a 2020 blog post from the author, announcing a pause of the project https://eli.sohl.com/2020/04/26/pressing-pause-on-theseus.ht...

rainsford3• 3 years ago

Elligator is both very cool and surprisingly less useful than one might expect. It does what it promises, but there really aren't all that many situations where it's really important to conceal the fact that you're exchanging Curve25519 keys.

The most common encrypted protocols advertise the cipher suite they're using, so there is little value in concealing the format of the keys being exchanged. And protocols trying to hide their existence entirely are pretty uncommon.

dang• 3 years ago

Year added. Thanks!

lenova• 3 years ago

Oooh interesting! I've been reading up on DHT's for the first time recently (I'm very late to the party).

It looks like the original inspiration was an anti-censorship network for sharing/providing scientific papers: https://eli.sohl.com/2017/02/17/theseus-robust-system-for-pr...

But as another poster noted, the project seems to be defunct do it being a solo effort: https://eli.sohl.com/2017/02/17/theseus-robust-system-for-pr...

I'm curious why the OP is interested in this project, or if anyone else has any interesting DHT tools they want to share?

kang• 3 years ago

libp2p: https://curriculum.pl-launchpad.io/curriculum/libp2p/dht/

wh33zle• 3 years ago

At the bottom is the link to the more technical specification: https://github.com/libp2p/specs/blob/master/kad-dht/README.m...

feross• 3 years ago

bittorrent-dht: https://github.com/webtorrent/bittorrent-dht (JavaScript implementation used by WebTorrent)

anacrolix5• 3 years ago

https://github.com/anacrolix/dht

ur-whale• 3 years ago

To a passive observer, all Theseus DHT protocol traffic is indistinguishable from random noise.

This.

And it should have been a design requirement for every protocol on the internet since day one.

earnesti• 3 years ago

I would have rather have every protocol to supply endless amount of ice cream.

sedatk• 3 years ago

So, identifying this protocol must be trivial: it's the one that looks like random noise.

rainsford3• 3 years ago

It's a cool property for sure, but in reality it's pretty difficult to implement, especially if you want a protocol that's extensible, interoperable, or allows open communication. At minimum, you're probably going to have to assume any participants trying to speak the protocol to each other have pre-shared cryptographic keys and/or protocol parameters.

Even a protocol like Wireguard for example, which does hardcode protocol parameters and requires the client to know the server's public key in advance, is not designed to be indistinguishable from random noise. You can certainly do it, but you lose a lot of other properties that are arguably more important to protocol design than indistinguishably.

psychphysic• 3 years ago

Can you explain this, how can it be indistiguishable from random noise? What does that even mean? And why is it even important/useful? Surely even if it looks like noise anyone can see that you are communicating with the DHT network?

eykanal• 3 years ago

This seems to be ~4 years old. Have any cryptographers reviewed / audited this protocol in the interim? A quick google search comes up empty [1], and google scholar doesn't show anything [2].

[1] https://www.google.com/search?&q=Theseus+dht+protocol+%22aud...

[2] https://scholar.google.com/scholar?q=theseus+dht+protocol

boramalper2• 3 years ago

IANAC† but I think the use of Noise Protocol is completely bogus. Regardless of your protocol of choice, you cannot encrypt things end-to-end if you cannot authenticate who is at the other end that you're encrypting for—hence the use of "safety numbers"[0] in Signal, "certificate authorities"[1] in TLS, "web of trust"[2] in PGP, and the list goes on and on...

This is at best an obfuscation attempt to make fingerprinting more expensive, just like BitTorrent protocol encryption[3].

† I Am Not A Cryptographer

[0] https://support.signal.org/hc/en-us/articles/360007060632-Wh...

[1] https://en.wikipedia.org/wiki/Certificate_authority

[2] https://en.wikipedia.org/wiki/Web_of_trust

[3] https://en.wikipedia.org/wiki/BitTorrent_protocol_encryption

sohle5• 3 years ago

Hi, I wrote this post. Let me use your comment as a chance to talk about what I was going for with the use of Noise here.

You're correct that part of the assumption inherent in end-to-end encryption is that you've got the other end right; otherwise, MitM attacks become possible. To prevent this, Social apps like Signal (also WhatsApp and many others) give you, roughly speaking, a way of confirming that both endpoints have negotiated the same secure channel. Less-social web apps use PKI. In both these cases, the design makes sense because you care who the other endpoint is. Their identity is meaningful to you outside of the context of the protocol.

In the peer-to-peer context, though, this identity information is mostly meaningless. I think, as long as there are no "network admins" or similarly privileged identities worth claiming, it's less important to certify identities up-front (what would even be a reasonable standard for accepting them?) and it's more important to be able to persist them. You want to make sure you're still talking to the same person you were talking to last time, whoever they are. This is what the PSKs are meant to accomplish (though they are not the only possible solution here).

This is technically still MitM'able, but only if you catch two parties on their very first connection; after that, they have a shared secret, and you should be locked out. Technically this falls slightly short of the ultimate goal of end-to-end encryption, but I still think it's a good fit for this context.

I went into this in (much) more detail in this old blog post: https://eli.sohl.com/2017/06/09/message-encryption-in-theseu...

kodablah• 3 years ago

I once combined libp2p/ipfs's DHT impl with Tor many moons ago: https://github.com/cretz/tor-dht-poc

orthecreedence• 3 years ago

Cool project, which lead me to this post: https://eli.sohl.com/2020/06/10/sybil-defense.html

I'm in this space right now (Sybil-resistant DHTs) for an identity project and finding things like this is absolute gold for me. Is there a good place to find more articles on modern sybil protection mechanisms (other than the proof-of-whatever blockchain bs)?

EGreg2• 3 years ago

See my sibling comment. I am looking for partners to help implement this “holy grail” stuff together

sohle5• 3 years ago

Hi, I wrote this post, and I just wanted to say thanks for the kind words - this research was a lot of fun to do and I'm glad it has an audience.

The "best" resources I know of are the ones I cited in that post, though do bear in mind it's a few years old. That said, I did want to make sure you didn't miss this companion piece: https://eli.sohl.com/2020/06/05/dht-size-estimation.html which is applicable to detecting (and - conjecturally - responding to) Sybil attacks.

h0p3• 3 years ago

Just wanted to say thank you, again. Your project gave me a lot to think about.

anacrolix5• 3 years ago

http://bittorrent.org/beps/bep_0042.html

EGreg2• 3 years ago

Anyone here an expert on DHTs, or maybe an enthusiast?

Back in 2018 when I was designing the Intercoin Protocol (which we now call the Intercloud) I met with Petar Maymounkov, who invented Kademlia in the early 2000s. Dude used to teach at NYU (my old school) so I emailed him and took him to Olive Tree Cafe — wound up asking him a bunch of questions and learning a lot.

Anyway, I have a question about all this. Consensus is an expensive operation and th reason Blockchains are slow is because they run fu network consensus cor every block. In Intercloud, we run network-wide consensus only occasionally, and mostly to update a provably random seed (by combining input from many different nodes). This serves as a seed for a random oracle for the network.

Now, that oracle is used during “shuffling”. I think we may have originally got the idea from the SAFE project. What happens is that, every so often, swarms of nodes are shuffled and nodes are reassigned to other swarms. So although they may pick their original swarm, they have no control over what future swarms they will be in, because it is effectively impossible for then to predict, let alone control, the random oracle and hashes of activity from neighboring swarms. Being off by a single bit throws off any of their well-laid plans.

Shuffling is necessary to break up swarms that have been corrupted, overrun by dishonest nodes sybil attacks, or just unable to reach consensus due to unavailability.

So most of these DHT issues go away. However, encryption is still a very good idea.

Another GREAT idea is what SAFE pioneered - removing the IP addresses from responses after the first hop. Because otherwise it is trivial to DDOS a network or a subset of it.

PS: last year I interviewed Ian Clarke (creator of Freenet) about this stuff https://m.youtube.com/watch?v=JWrRqUkJpMQ

anacrolix5• 3 years ago

I like the ideas in this, and that it's got changes that allow it to work over stream based protocols, which means webrtc and tor. However DHTs (Kademlia at least) aren't efficient on a large scale if they're not built on packet oriented protocols. I don't know what the solution is, maybe allowing TCP is just a compromise you have to make sometimes.

sohle5• 3 years ago

Hi, I wrote this post, and this is something I was wondering about the whole time. It seemed like TCP was a necessary compromise, but I wasn't sure how painful the overhead would be. If there are any notable examples you know of where this went wrong (or right) I'd be interested.

KirillPanov• 3 years ago

If there are any notable examples you know of where this went wrong

IPFS

Stop using sessionful protocols for DHTs. It. does. not. work.

anacrolix5• 3 years ago

If it helps, you may find that when you scale your DHT to 10000+ nodes you will start to have issues with building on TCP. Of course many DHTs never get to that scale. IPFS is an example of one that sits just above that mark and can't figure out why things don't work well.

For comparison, the BitTorrent DHT has 20 MILLION nodes and works fantastically.

unstruktured• 3 years ago

A more recent attempt at a DHT by the Holochain team: https://developer.holochain.org/concepts/4_dht/

Theseus DHT Protocol (2018)

Related Stories