apenwarr

I’m glad you liked it!

Japanese manufacturing addressed this too! There’s a whole branch of Deming’s work around supply chains and how it’s often worth working with “more expensive” suppliers when their quality is consistently high, because compensating for low quality inputs to your own work is far more expensive.

(Tailscale founder here) Two main differences: first, every DERP server used by your tailnet must be accessible by every node on your tailnet at all times, otherwise you get hard-to-debug netsplits. That's a very high bar to maintain so we've historically recommended you don't try. In contrast, peer relays are "if a given pair of nodes can connect through any of the relays, go for it" so deploying one is always a performance and reliability improvement.

Secondly, peer relays support UDP while DERP is TCP-only. That would be fixable by simply improving the DERP protocol, but as we explored that option, we decided to implement the Peer Relay layer instead as a more complete solution.

[Tailscale CEO here] I see a lot of comments asking why Tailscale would branch away from our "core product" and build this thing that seems unrelated at first. One answer is that just about every single Tailscale customer (or homelab user!) is dipping their toes into AI right now, and they often come to us and ask how to integrate their stuff into Tailscale. Aperture is our answer to that.

A separate goal I have personally: demonstrate that anyone can build really neat stuff directly on top of the "Tailscale platform." One of my rules for the Aperture team was, you're not allowed to change core Tailscale, you have to build entirely on top as if you were some partner company. So this is a demo of how anybody can make pretty slick, easy-to-use, and yet highly secure stuff by building on Tailscale (the open source packages, or the commercial product, or both).

(I'm a Tailscale employee) The recent versions of the Tailscale k8s operator actually used a pre-release of the Services feature to do exactly that. So, not much difference. The official Services release is making that functionality available for more use cases (and generally better documented and user friendly).

You’re right except in the very specific case of the App Store purchase or download process. You only get one chance at FaceID and then it demands a password. But, if you cancel and do it again, you get another chance at FaceID. It’s mystifying why they’d make that UX choice.

(Tailscale cofounder here) Tailscale already gives every node on every tailnet a globally unique internal IPv6 address, that is reachable even if you don't have IPv6 on the "outside" network. If your apps and OSes are all willing to use IPv6, you haven't had a problem since the early days of Tailscale; they've been solved for years.

Alas, the "apps and OSes are all willing to use IPv6" problem is a persistent one, so we have to make IPv4 work too.

It’s possible to build a cache without any insight, but to make a cache that works well in a given domain requires knowledge from outside the world of caching itself. The same is true of summarization and compression.

Tailscale now supports custom OIDC providers. But if you already have the ability to host one, you won’t benefit from what’s in the above article (which is about hosting stuff at home even without a public IP address). https://tailscale.com/blog/custom-oidc/

I recommend against using Funnel for this use case (because it exposes your server to everyone in the world, not just your trusted users). Tailscale node sharing is free and secure for private networks of friends, and there are lots of people using it with Minecraft: https://tailscale.com/kb/1084/sharing/

To answer the question in another thread, node sharing also works with UDP. (Funnel is TCP-only due to the vagaries of IP addresses and TLS certificates when facing the outside world, sigh.)

(Interviewee here) WireGuard itself barely touches DNS, and tailscale as far as I know doesn’t have any code that would change how your external DNS resolution works. What you’re seeing might be a misdiagnosis. If you email tailscale support we’ll be happy to help figure out what happened and if it’s a bug.

Exit nodes are usually still behind your firewall and have no open incoming ports. If you're willing to reconfigure your firewall to open incoming ports, you probably didn’t need Funnel in the first place.

I haven't played with OpenWRT SQM for a while, but if it's easy to reproduce with gfblip, it might be a simple matter of telling them your exact SQM settings and the URL to try.

Chances are changing the SQM backend (eg. between fq-codel and cake) will at least change the behaviour and likely make the bug go away.

In general, the load generated by a series of these pings is so low as not to matter, unless a whole ton of people start doing it at once. But in that case, gfblip's trivial backend code will ask the frontends to slow down so that aggregate load stays low.

I've been on the Internet for a long time, including doing Debian stuff (popularity contest!) back in the 1990s. I don't know what attrition.org is though.

Oh neat! If you see occasional red blips, you have probably found a bug (or misconfiguration) in your OpenWRT SQM. gfblip doesn't lie. In fact, I created it to find annoying router bugs like this. If you click the "DNS" checkbox you can find all sorts of terrible router bugs.

It's not too surprising that you don't see latency increase under light load like Youtube. "Under load" really means uploading (especially) or downloading (sometimes) at maximum speed. But Youtube usually doesn't download at maximum speed, because that would imply you can't keep up with its streaming video rate, and you'd get glitches. So your connection is likely not really "under load" at that time. Try uploading a large file somewhere and you should see an immediate change in blips.

However, the preflight check is sufficient to check latency :)

"Preflight check" is such a wrong analogy, since with CORS you fly all the way to the destination to check if you're allowed to fly to the destination.

You might mean if you turn "on" HTTPS-only mode?

The silly trick blip uses involves pinging non-encrypted HTTP web servers, which is not allowed from an encrypted web page. So you really have to load blip from a non-encrypted server.

This code is from 10 years ago. Most likely we could find some way to work with HTTPS nowadays, but alas, I don't have time to maintain it.

When you ping to the phone, you're probably running into the phone's power saving feature. When the phone generates the request, it stays powered up for a bit until the answer comes back, which is why you're not seeing the same random delay from blip.

"https with auth" is fine and good, and obviously the world has been heading in this direction. But I secretly suspect this is because 90%+ of developers nowadays don't know how to hack on any layer below http.

Tailscale is not a typical VPN; it's just a system that attempts to provide beyondcorp-like behaviour at a lower level of the stack, so that you don't have to rewrite all your apps (ssh in this case!) to use https, and don't have to have open ports in your firewall, and don't have to run everything through the cloud if you don't want.

As in my post above, there's more than one way to do it. You can also build traditional-beyondcorp-over-https on top of a Tailscale network, so you get all the improved network connectivity and also all the benefits of a "pure" beyondcorp architecture.

(I'm a tailscale cofounder) I think of Tailscale more like a set of tools that lets you do any architecture you want. Nobody has to use Tailscale ssh console, but if you believe in the future of wasm -> apps -> web console -> ssh, now you can have it.

On the other hand, if you believe in the future of OS private network connectivity -> console -> ssh, then you had that already with native Tailscale and Tailscale ssh.

If you believe in OS private network connectivity -> browser -> javascript console -> ssh, then you can do that too, by installing tailscale in the native OS and then the browser can use it.

I actually agree with you, I'm very suspicious about a world where we just move everything into the web browser. But on the other hand, sometimes it's really handy to have that option.

Alas, the real “telnet” protocol has considerably more fanciness than nc. It’s just that the telnet cli command degrades into a simple line-oriented mode if it doesn’t see the telnetd init sequence.

Note that you can create a (public) Github org nowadays for free, then use it with Tailscale. This is a pretty smart move on Github's part.

(I wrote the article.) You're not that unusual, we just haven’t had time to address that use case directly yet. I expect an ecosystem of MSPs may arise to offer physical boxes, or some such thing, since the tailscale client is open source. (Or you could buy a Synology with tailscale on it I suppose!)

Many companies just run tailscale in a VM to replace their physical VPN concentrator boxes.

This article was me trying to write down my internal notes on the topic at the time, after having dived very deeply into problems with the set of interlocking network queues inside embedded wifi devices we were working on.

I was probably not the first to realize that this set of rules probably applies to all kinds of networks, not just ones in embedded device firmware, but I hadn’t really seen it written down before.

But I’m surprised the article ever made it to the HN front page, even 5 years later, since it doesn’t even attempt to address the how/why/how do you know sorts of questions. It’s mainly a placeholder just so I don’t forget. (Those rules for muxes and demuxes and backpressure are really confusing, but I believe them to be strictly correct.)

The rule about bottlenecks (there is only ever one) I borrowed from the TCP BBR paper. The rule about queues on a path always being empty except for exactly one that is always full, I think I borrowed from a talk by Stuart Cheshire that I can never seem to find when I look.

You might like the Tailscale Machine Key approval feature. If enabled for your domain, no device can join your tailscale network unless they’re manually approved by your network’s admin or the API.

Someone else mentioned that basically malware on your machine is expected to bypass all security layers. So we’re basically saying “the ‘new’ network layer is now as trustworthy as the app layer” rather than claiming everything is perfect, you’re right.

This is one reason we limited taildrop to only transfer between devices owned by a single user for now, and only to drop files into controlled locations. Tailscale also has ACL policies for when you don’t trust all the endpoints to just do anything.

The localapi is indeed an http server built into the tailscaled process, which is written in Go. Since we already had an http client in there, the net new code to add an http server is quite low. And it doesn’t take any battery unless it’s being queried. I think people underestimate how cheap http can be (once you’ve paid the up front cost, anyway).

You’ve correctly guessed that blog link [0] that explains the reason I don’t think we’d ever want to try making a distributed coordination service. Most importantly, corporate customers absolutely love having a single control and registration point for every corporate authorized device on their network (and thus, a way to instantly deauthorize stolen devices). What we’re going to do though is add private audit trails and tamper proofing, kind of like TLS certificate transparency, so that the central instructions can be validated in a decentralized way, if that makes sense. More on that later. :)

Re: peerapi, there are lots of ways to build app layer protocols once you have tailscale making the connection itself easier. We picked http since it was the fewest lines of code and it makes an easy example.

Re: live video, Jitsi already works fine on a tailscale network if you want to try that.

There’s an open source project called headscale (not written by or officially supported by tailscale’s team, but we like it) which you can point tailscale’s clients at. Then your whole system is open source. You can also avoid using a central IdP that way, if that’s what you want. (I strongly recommend caution about that, if you want good security. I know it’s not popular to say so on HN, but most people running their own IdP will do it less securely than one of the big providers. It’s a very hard job, akin to running a root CA.)

MacOS, iOS, and Linux clients can use your native OS updates. Windows needs to be updated by hand or with something like chocolatey or MDM. But more importantly, we have a policy of not breaking old clients if we can possibly avoid it. So far we have never deprecated old clients. We extend our protocols in a backward compatible way, because unilaterally breaking your network infrastructure… really sucks.

The way tailscale networks (tailnets) work is probably not how you’re used to thinking about them. Each node has its own view of the world, based on which nodes and services are shared with it in particular. We have security policy settings per domain, and a node sharing UI that lets you share any of your devices with anyone else.

The default model is that all devices belonging to someone in the same domain, say tailscale.com, can see each other. But we’re working on making that even more flexible since it doesn’t always do what you want for huge orgs (like universities).