Decoupling IP from services and sockets for addressing agility

https://www.micahlerner.com/2022/01/13/the-ties-that-un-bind-decoupling-ip-from-web-services-and-sockets-for-robust-addressing-agility-at-cdn-scale.html

by mlerner • 4 years ago

36 7 4 years ago

micahlerner.com

chatmasta• 4 years ago

About a month after this blog post was published, Cloudflare open sourced a tool called Tubular for binding sockets with eBPF. Nobody really noticed. But it’s extremely cool and it works.

Blog: https://blog.cloudflare.com/tubular-fixing-the-socket-api-wi...

GitHub: https://github.com/cloudflare/tubular

metadat• 4 years ago

Agreed, for whatever reason it never got much attention.

https://news.ycombinator.com/item?id=30375516 (2 comments)

ook• 4 years ago

https://research.cloudflare.com/publications/Fayed2021/ has a link to download the full pdf of the paper.

0xbadcafebee2• 4 years ago

One of the things I want out of a new TCP/IP stack is being able to be more explicit about what protocol to use and how to route a request. Example:

  !dns+ipv6://github.com>org>yc!tcp+tls+http+restapiv3://users/list

Here's what that says:

1. Tell the application to use DNS and IPv6 to connect to the host "github.com". github.com sees a packet with that request in it, sees ">org" after "github.com", and so forwards the packet to host "org". That host "org" gets the packet, sees ">yc", and forwards the packet to host "yc". So the client has only sent one packet, but the server is intelligently routing the request - with only IPv6 Routing and DNS - to a backend set of servers/load balancers. No round trip yet.

2. Once that IP routing is done, and the packet gets to "yc", now it can start unfolding the rest of the packet. It sees "tcp", so, okay, let's do TCP; "tls", so let's open a TLS connection; "http", so, okay, this is an HTTP request; and "restapiv3" tells the host to look up a service called "restapiv3", which some application on some port is listening on; this does not need to be a static port like 443, it is looking up the name "restapiv3" in a table and will connect to the correct file descriptor/socket/etc, and that application's socket will be a tcp socket, the app will talk tls, http, etc. No need for hard-coded port numbers.

3. Once that service is connected to, request the "users/list" resource at that REST API. Packets are routed by each of the previously mentioned backend hops to establish the connection. The client sees it is connected to "github.com>org>yc:tcp+tls+http+restapiv3", although without DNS it could be "1.2.3.4>192.168.3.5>172.16.0.3:tcp+tls+http+restapiv3".

That gives us:

1. Ability to decouple protocols. Don't want to use DNS? Want to use a new version of DNS? Or IP? That's fine, just mention it in the stack address field. Want to use TLS but don't want to use HTTP? That's fine too, it won't matter to anything except the final service you're routing to, as border load balancers do not need to speak application protocols, they are literally just routers.

2. Intelligent routing into networks that might otherwise be unroutable. This allows more complex network topologies that aren't limited to public network address space, and it means large corporate networks don't need to carefully manage their internal IP space. It also means we don't have to unpack network connections at a border load balancer, so we could for example have true end to end encryption from a client to a backend server with no intermediate servers sniffing traffic.

3. Stack an unlimited number of protocol frames into one URI, allowing you to bundle stacks in stacks, apps in apps, etc. Everything is routed and unpacked in the background by the tcp/ip stack.

4. No more port numbers for services, and routing to logical service names on backend hosts, means we aren't forced to use HTTP as a universal transport for every new protocol. HTTP is used mainly to route connections because we are stuck with one port number (443) for every application-layer protocol. WebSockets exists because of that too, since there's no other way to open a bidirectional connection to a backend.

5. Intelligent network troubleshooting. An error in a connection could report on exactly what hop and layer and protocol failed the connection at.

ninefathom• 4 years ago

You just described, almost without discrepancy, X.25.

That does not necessarily mean anything or diminish the value of what you're suggesting; it is simply worth noting that this has been done before (and fell by the wayside decades ago).

0xbadcafebee2• 4 years ago

I doubt X.25 abstracts away L4 and L7! You still need port numbers so you still need HTTP for everything, application unpacking at the load balancer, a service mesh, etc. X.25 seems to end at the network layer.

icedchai• 4 years ago

It also sounds a bit like email bang paths. In the old (UUCP) days, you could manually route your email: host1!host2!host3!joe

Decoupling IP from services and sockets for addressing agility

Related Stories