euank

I can give an n=1 anecdote here: the dns resolver used to have hard-coded caching which meant that it would be unresponsive to pod updates, and cause mini 30s outages.

The code in question was: https://github.com/grpc/grpc-go/blob/b597a8e1d0ce3f63ef8a7b6...

That meant that deploying a service which drained in less than 30s would have a little mini-outage for that service until the in-process DNS cache expired, with of course no way to configure it.

Kuberesolver streams updates, and thus lets clients talk to new pods almost immediately.

I think things are a little better now, but based on my reading of https://github.com/grpc/grpc/issues/12295, it looks like the dns resolver still might not resolve new pod names quickly in some cases.

Coincidentally, Google has sorta done that too!

ChromiumOS/ChromeOS is built using portage, and is certainly rolling release. It's not really a traditional linux distro, but it's something.

Does anyone know if and where I could file a bug report?

The linux kernel does take bug reports: https://docs.kernel.org/admin-guide/reporting-issues.html

However, that bug probably isn't specific enough as you've described it, unless you can find the commit causing it (such as via a git bisect https://docs.kernel.org/admin-guide/bug-bisect.html), or come up with a clearer repro.

Alternatively, if you're seeing the issue on a distro-maintained kernel (such as on fedora/ubuntu/debian with their kernel package), reporting the issue to the distro maintainers may be more appropriate.

Email has never had explicit threading

The "In-Reply-To" header is described in rfc2822. It is an explicit header in the RFC that is how you create threads.

Every mail client I've used correctly understands how to thread reply-chains using In-Reply-To.

The thing you're talking about, steam receipts grouping, is not a feature of email, but a specific feature of gmail's web view which is not mandated by any RFC and indeed is not explicit threading...

But there is a real way to thread which is defined in the RFC, and if you use a reasonable email client (aka not gmail), every mailing list's threading will work for you.

The kubelet can talk to containerd's cri endpoint, yes, but there's one additional bit of complexity.

If someone wants to use kubelet + docker so that they can, for example, ssh into a node and type 'docker ps' to see containers, or have something else using the docker api see the containers the kubelet started, that won't work after re-pointing the kubelet from docker to containerd.

The difference here is namespacing[0], but not the linux-kernel-container-namespace, rather the containerd concept by the same name to allow "multi-tenancy" of a single containerd daemon.

In addition, I don't think you could have docker + cri run in the same containerd namespace since they end up using different networking and storage containerd plugins. I think that terminology is right.

So yeah, repointing the kubelet to containerd directly works fine, but it won't be the same thing as running docker containers.

[0] https://github.com/containerd/containerd/blob/9561d938/docs/...

For what it's worth, there are a few cases where docker vs some other runtime does make a difference.

One difference is that if you 'docker build' or 'docker load' an image on a node, with docker as a runtime a pod could be started using that image, but if containerd is the runtime it would have had to be 'ctr image import'ed instead.

I know that minikube, at some point, suggested people use 'DOCKER_HOST=..' + 'docker build' to make images available to that minikube node, which this would cause to not work.

It would be nice if k8s had its own container image store so you could 'kubectl image load' in a runtime agnostic way, but unfortunately managing the fetching of container images has ended up as something the runtime does, and k8s has no awareness of above the runtime.

Oh, and for production clusters, a distribution moving from dockerd to containerd could break a few things, like random gunk in the ecosystem that tries to find kubernetes pods by querying the docker api and checking labels. I think there's some monitoring and logging tools that do that.

If distributions move from docker to docker-via-a-cri-shim, that won't break either of those use cases of course.

containerd can serve CRI requests itself. This has been the case since the containerd v1.1.0 release[0], which included the cri "plugin" as an in-process part of the containerd binary. For a while, to keep up the plugin idea, it was in a separate github repo too, but these days it's in the main containerd repo directly[1].

[0] https://github.com/containerd/containerd/releases/tag/v1.1.0

[1] https://github.com/containerd/containerd/tree/9561d9389d/pkg...

I think that the title of this is a bit misleading.

Kubernetes is removing the "dockershim", which is special in-process support the kubelet has for docker.

However, the kubelet still has the CRI (container runtime interface) to support arbitrary runtimes. containerd is currently supported via the CRI, as is every runtime except docker. Docker is being moved from having special-case support to being the same in terms of support as other runtimes.

Does that mean using docker as your runtime is deprecated? I don't think so. You just have to use docker via a CRI layer instead of via the in-process dockershim layer. Since there hasn't been a need until now for an out-of-process cri->docker-api translation layer, there isn't a well supported one I don't think, but now that they've announced the intent to remove dockershim, I have no doubt that there will be a supported cri -> docker layer before long.

Maybe the docker project will add built-in support for exposing a CRI interface and save us an extra daemon (as containerd did).

In short, the title's misleading from my understanding. The Kubelet is removing the special-cased dockershim, but k8s distributions that ship with docker as the runtime should be able to run a cri->docker layer to retain docker support.

For more info on this, see the discussion on this pr: https://github.com/kubernetes/kubernetes/pull/94624

is there a name for this bias?

I think it's response bias. https://en.wikipedia.org/wiki/Response_bias

You're right that nixos / all nix packages isn't/aren't perfectly reproducible.

In practice, most of the packages in the nixos base system seem to be reproducible, as tested here: https://r13y.com/

Naturally, that doesn't prove they are perfectly reproducible, merely that we don't observe unreproducibility.

Nix has tooling, like `nix-build --check`, the sandbox, etc which make it much easier to make things likely to be reproducible.

I'm actually fairly confident that the redis package is reproducible (having run `nix-build --check` on it, and seen it have identical outputs across machines), which is part of why I picked it as my example above.

However, I think my point stands. Dockerfiles make no real attempt to enforce reproducibility, and rarely are reproducible.

Nix packages push you in the right direction, and from practical observation, usually are reproducible.

A Dockerfile is not a reproducible set of build instructions in most cases. I'd guess that the vast majority of Dockerfiles are not reproducible.

Let's look at an example dockerfile for redis (based on [0])

    FROM debian:buster-slim
    RUN apt-get update; apt-get install -y --no-install-recommends gcc
    RUN wget http://download.redis.io/releases/redis-6.0.6.tar.gz && tar xvf redis* && cd redis-6.0.6 && make install

The unreproducible bits are the following:

1. FROM debian:buster-slim -- unreproducible, the base image may change

2. apt-get update && apt-get install -- unreproducible, will give a different version of gcc and other apt packages at different times

Those two bits of unreprodicble-ness are so core to the image, that they result in every other step not being reproducible either.

As a result, when you 'docker build' that over time, it's very unlikely you'll get a bit-for-bit identical redis binary at the other end. Even a minor gcc version change will likely result in a different binary.

As a contrast to this, let's look at a reproducible build of redis using nix. In nixpkgs, it looks like so [1].

If I want a reproducible shell environment, I simply have to pin down its dependencies, which can be done by the following:

    let
      pkgs = import (builtins.fetchTarball {
        url = "https://github.com/NixOS/nixpkgs/archive/48dfc9fa97d762bce28cc8372a2dd3805d14c633.tar.gz";
        sha256 = "0mqq9hchd8mi1qpd23lwnwa88s67ac257k60hsv795446y7dlld2";
      }) {};
    in pkgs.mkShell {
      buildInputs = [ pkgs.redis];
    }

This is true of the majority of nix packages. All commands are run in a sandbox with no access to most of the filesystem or network, encouraging reproducibility. Network access is mediated by special functions (like fetchTarball and fetchGit) which require including a sha256.

All network access going through those specially denoted means of network IO means it's very easy to back up all dependencies (i.e. the redis source code referenced in [1]), and the sha256 means it's easy to use mirrors without having to trust them to be unmodified.

It's possible to make an unreproducible nix package, but it requires going out of your way to do so, and rarely happens in practice. Conversely, it's possible to make a reproducible dockerfile, but it requires going out of your way to do so, and rarely happens in practice.

Oh, and for bonus points, you can build reproduible docker images using nix. This post has a good intro to how to play with that [2].

[0] https://github.com/docker-library/redis/blob/bfd904a808cf68d...

[1] https://github.com/NixOS/nixpkgs/blob/a7832c42da266857e98516...

[2] https://christine.website/blog/i-was-wrong-about-nix-2020-02...

This proposal is more powerful than that one rust macro, but rust's abilities around embedding files are much more powerful than go's approach.

This proposal allows "go build" to embed things in a very specific way, but it's not meant to be extensible.

Rust's 'include_bytes!' macro on the other hand is a macro in the stdlib that can be emulated in an external library. I'm fairly sure every feature of go's embed proposal could be implemented via a rust macro outside the stdlib.

For a specific example, I had a project where I wanted to serve the project's source code as a tarball (to comply with the AGPL license of the project). I was able to write a rust macro that made this as easy as effectively "SOURCE_TARBALL = include_repo!()" [0] to embed the output of 'git archive' in my binary.

Of course, there's a very conscious tradeoff being made here. In rust, "cargo build" allows arbitrary execution of code for any dependency (trivially via build.rs), while in go, "go build" is meant to be a safe operation with minimal extensibility, side effects, or slowdowns.

[0] https://github.com/euank/include-repo

Go's channels aren't exposed outside a specific go process's runtime. The runtime doesn't give you any convenient way to redirect them. They're not like erlang's mailboxes at all in that regard.

Furthermore, channels aren't the primitive used for multiplexing IO / handling connections on a socket in go. You typically have a goroutine (e.g. 'http.ListenAndServe' spins up goroutines), and the gorutines are managed not by channels, but by the internal go runtime's scheduler and IO implementation (which internally uses epoll).

Because of all those things, replacing a running go process that's listening on sockets is no different from that same problem in C. You end up using SO_REUSEPORT and then passing the file-descriptors to the new process and converting them back into listeners. Channels don't end up factoring into it meaningfully.

If you're interested in what this looks like, cloudflare wrote a library called tableflip [0] which does this. I also forked that library [1] to handle file-descriptor handoff in a more generic way, so I've ended up digging pretty deeply into the details of how this works in go.

[0] https://github.com/cloudflare/tableflip

[1] https://github.com/ngrok/tableroll

One of the weird peculiarities of path-style API requests was that it meant CORS headers meant nothing for any bucket pretty much. I wrote a post about this a bit ago [0].

I guess after this change, the cors configuration will finally do something!

On the flip side, anyone who wants to list buckets entirely from the client-side javascript sdk won't be able to anymore unless Amazon also modifies cors headers on the API endpoint further after disabling path-style requests.

[0] https://euank.com/2018/11/12/s3-cors-pfffff.html

It does not currently (only linux and macOS), but porting it over should be really easy!

The cross platform story for rust isn't as seamless as an interpreted language like lua, but it's still good enough that it's not a huge effort to support another platform or shell.

cross platform/shell support

Shell support should be fairly similarly easy or hard between most autojump implementations.

Fair question! I didn't know a rust rewrite of autojump existed!

I already benchmark pazi against autojump by itself, so adding in a benchmark against autojump-rs will basically be drop-in, and I do expect pazi will still be faster, but I'll see.

At a glance, the differences I see are the following:

* pazi has extensive integration tests and benchmark code

* pazi doesn't fork processes into the background like autojump's shell scripts do (and therefore autojump-rs does), which avoids a few races

* pazi's interactive selection (z -i) makes use of the terminal's alternate buffer to avoid blowing away your scrollback, which most jumpers don't bother with.

* pazi has a cooler name in my opinion :)

These differences are not all that major. pazi is definitely short on some features, and I don't blame you if you wish to use one of the others. I'll keep working on pazi because I like using it and I'm having fun working on it / optimizing it / testing it.

This looks nice! I'm especially interested in the claims of performance.

I've also written an autojumper[0] (in rust) after being frustrated by performance issues in other autojumpers, among other reasons.

For mine, I've gone to the effort to write a test and benchmark harness, and I also ended up with about 10x faster than fasd/autojump[1].

I'm excited to benchmark pazi against z.lua, and if needed take some ideas to be even faster :)

[0] https://github.com/euank/pazi

[1] https://github.com/euank/pazi/blob/v0.2.0/docs/Benchmarks.md...

I can recommend pazi as a convenient tool for zapping around https://github.com/euank/pazi

The readme also lists several alternatives in the same space as pazi.

I've found fasd (https://github.com/clvv/fasd) to be the most featureful of the directory jumping tools, and it's much faster than autojump as well.

I'd be remiss not to mention pazi however, which is the tool I wrote to replace fasd in my own workflow: https://github.com/euank/pazi

It's similar to autojump, but much faster: https://github.com/euank/pazi/blob/master/docs/Benchmarks.md...

Oh, and it's written in rust :)

This is an anomaly that your explanation doesn't apply to though.

Notice that it has 50 points and is half an hour old. I count 12 items on the front page currently that have 50 or fewer points and are 1 or more hours old and yet remain there. These items are generally quality content (such as the tinfoil security post -- https://news.ycombinator.com/item?id=8086834 -- which sits at 37 points / 3 hours).

Your explanation of "5 votes within the first 30 minutes" completely disregards that this has as many votes as many other items on the front page, but in a fraction of the time period.

This is an interesting oddity such as you might expect to see on reddit's /r/mildlyinteresting or til, but I don't feel that this fits well on HN personally. I'm not saying HN is turning into reddit by having this content, just pointing out how I'd classify this content.

This is absolutely not "intellectually gratifying" in my mind. The fact that it can be trivially consumed (as opposed to a long blog post) might in part lead to it having a higher number of upvotes due to being more accessible, but that doesn't mean that it's better than a less accessible item that gets more upvotes relative to people consuming it, but fewer as an absolute.

Edit: It was 1st for about two minutes and then dropped down to 198 instantly (among items from 1-3 days ago and/or with over 20 times fewer points). I'll hazard a guess that the mods agree with me on this and nuked it down.

They don't mention Amazon specifically except in the title and the final sentence.

I recall, when I signed up for my free year of AWS, that it required a valid credit card. They claimed to only target providers that need essentially only an email address.

I think that detail makes the title incredibly misleading.

"show on the right" required a restart after being checked to work for me in linux. Perhaps that helps?

It was the parent I responded to who said all software should strive for this. My argument is not against the philosophy of the window manager in this instance, but rather against applying it everywhere as the parent seemed to think would be good.

So no, you can't really compare those, I agree, but it was not me who brought in an arbitrary piece of software, but the parent comment.

The fraction of terrorists in "all people that use the internet" is approximately equal to the fraction of terrorists in "privacy seekers" -- both are roughly zero.

The NSA only was given such pervasive power to catch terrorists. The fact that privacy seekers are more likely to be "bad actors" is moot unless that means terrorists because regular "bad actors" are supposed to be innocent until guilty, handled by police/fbi, etc. NSA only is allowed to work essentially with no due process because it's for cathing "terrorists". Interestingly, now that terrorists know about the NSA, they no doubt will simply not use the internet (or phones) at all, thus making it so the NSA can't catch a terrorist by any of its methods.

I definitely expect all terrorists are extremely careful about internet activity now that they know the NSA is so invasive, thus making the NSA's actions even less defensible (not that I thought they ever were).

I do not like the "API should be completely transparent" and "API must not be bloated".

I personally think APIs exist to hide complexity. They should make sense, they should be usable, but they need not be transparent.

Let's have a rather contrived example. In SDL, you can call SDL_Init(). That does a lot of things, and what it does differs on windows, linux, and OSX. I absolutely don't want the more transparent API of SDL_Init_Windows_Video(), SDL_Init_Windows_Audio(), SDL_Init_Linux_Video() .... And so on. Realistically, for absolute transparency, you'd just do away with the API and write every line of code SDL currently hides (Reductio ad absurdum I know)

The entire point of having that one SDL_Init() call that's identical on all platforms, even though it does many opaque things underneath (and different things per platforms) is that it's easier to understand and use. The goal of an API is to be usable and useful, nothing else. Transparency can often get in the way of usability.

As for the "bloated" bit. Well, I'd rather be able to do too many things than too few. It's a great goal to have a purely orthogonal (no method may be accomplished with another combination of methods) API, but it should not be done at the expense of usability.

    > Nothing should be put into it except what's impossible or impractical to do in pure Lua, and what's extremely common and likely to be used in everyone's configs.

This philosophy would lead to twitter not having a friendship API because the only thing practically every api user does is login and (arguably) tweet, only a few deal with friendships or account settings. It would lead to linux not having a configuration option for running more than one physical CPU or hotswapping ram because the wide majority of people will never use these options.

I'm happy that the author of this software decided to make it simple and it seems to be working here, but in the majority of cases of API design, I'd much rather the API be usable, not transparent, and large, not minimalistic. I'd rather be able to call a method that does some common activity for me which I could technically do myself in a 1000 lines of code than not have such a method because it's "bloat".

Jshint still complains about it even though it's technically okay.

Promises show a good use case. It's common to have many promise-laden bits of code and it's clear that the first callback is success, second, error, and third progress.

Having to do "def promise1_success" ... "def promise1_error" ... etc all the way up through 5 promises is a pain.

For event-emitter like things, it's also great to have anonymous functions.

For example, in nodejs, you might have reader.on('data', function(chunk) {}); .... You gain no further insight into what that function does if you have "def handle_reader_data" and then use it once there. There are many cases where you don't need the function name for clarify, and it just clutters the current scope to have it not be anonymous.

Code is also easier to follow if functions like that, used once only as a callback, are immediately defined where they are used, not lines above. When reading code that has a named function above, I'd have to skim the function the first time, see it used, and then trace back through it above knowing what was calling it.

Why no mention of FirefoxOS?

The Mozilla foundation has our best interests at heart much more than Google and while it's not C++, html5/javascript is certainly an open enough platform which basically everything has started to target anyways.

Sure, a pure linux phone would be cool in some ways, but FirefoxOS seems much closer, realer, and every bit as awesome.

I'm hoping it means the installation is non-destructive and it uses your existing configs, with no changes, as a base and then modifies them from there when you use the webui.

If that's true, it's poorly phrased.