thatcks

I am the parent, and any competent fsck should find these issues. Fsck traditionally explicitly verifies claimed free space against actual free space, and in a filesystem with ACLs it should also verify that filesystem level metadata like ACLs is sane, just as fsck verifies (for example) sane inode flags and inode field values. ZFS scrubs explicitly do not verify spacemap consistency or a lot of other sorts of consistency.

ZFS scrubs don't even verify that a filesystem's directory tree is acyclic and can reach every claimed in-use filesystem object, but I'm not aware of ZFS bugs in that area. This is because ZFS's 'metadata' for scrubs is much different than how it works in a traditional filesystem. To phrase it in conventional filesystem terms, ZFS has a big list of all in-use inodes, and it verifies the filesystem checksums by going through this list. The 'filesystem metadata' that a scrub verifies is the structure of this list of in-use inodes, plus some other things around the corners.

Two examples that I can find are https://github.com/openzfs/zfs/issues/7910, where very old versions of ZFS appear to have quietly written slightly incorrect ACL information, and https://bugs.launchpad.net/ubuntu/+source/zfs-linux/+bug/190... where Ubuntu 21.10 shipped with a bug that created corrupted ZFS filesystems. I believe https://www.illumos.org/issues/9847 may be another example of this, although less severe, where ZFS leaked disk space under some circumstances.

The article is correct but it downplays an important limitation of ZFS scrubs when it talks about how they're different from fsck and chkdsk. As the article says (in different words), ZFS scrubs do not check filesystem objects for correctness and consistency; it only checks that they have the expected checksum and so have not become corrupted due to disk errors or other problems. Unfortunately it's possible for ZFS bugs and issues to give you filesystem objects that have problems, and as it stands today ZFS doesn't have anything that either checks or corrects these. Sometimes you find them through incorrect results; sometimes you discover they exist through ZFS assertion failures triggering kernel panics.

(We run ZFS in production and have not been hit by these issues, at least not that we know about. But I know of some historical ZFS bugs in this area and mysterious issues that AFAIK have never been fully diagnosed.)

I can't remember the 10G firewall figures we got in testing off the top of my head, but we didn't max out the 10G network; I think we were getting somewhere in the 8G range. This is significantly better than our OpenBSD performance but not quite up to the level of full network speed or the full speed that two Linux machines can get talking directly to each other over our 10G network. I also suspect that performance depends on which specific NICs you have due to driver issues. The live performance of our deployed FreeBSD firewalls is harder to assess because people here don't push the network that hard very often (although every so often someone downloads a big thing from the right Internet source to get really good rates).

(I'm the author of the linked-to article.)

The short version is that we wound up not feeling particularly enthused about OpenBSD itself. We have a much better developed framework for handling Ubuntu machines, making it simply easier to have some more Ubuntu machines instead of OpenBSD machines, and we also felt Bind on Ubuntu was likely to be better supported than a ports Bind on OpenBSD. If everything else is equal we're going to make a machine Ubuntu instead of OpenBSD.

The computers that moved from OpenBSD to Ubuntu were our local resolving DNS servers. These don't use PF and we also wanted to switch from our previous OpenBSD setup to Bind, where we were already running Bind on Ubuntu for our DNS master servers. The gory details were written up here: https://utcc.utoronto.ca/~cks/space/blog/sysadmin/UsingBindN...

We may at some point switch our remaining OpenBSD DHCP server to Ubuntu (instead of to FreeBSD); like our DNS resolvers, it doesn't use PF, and we already operate a couple of Ubuntu DHCP servers. In general Ubuntu is our default choice for a Unix OS because we already run a lot of Ubuntu servers. But we have lots of PF firewall rules and no interest in trying to convert them to Linux firewall rules, so anything significant involving them is a natural environment for FreeBSD.

(I'm the author of the linked-to article.)

There is no 'RAW' format as such. In practice, 'RAW' is a jargon term for "camera specific format with basically raw sensor readings and various additional information". Typically the various RAW formats don't embed the spectral information, just a camera model identifier, because why waste space on stuff the camera makers already know and will put in their (usually maker specific) processing software.

(Eg Nikon's format is 'NEF', Canon's is 'CR3', and so on, named after the file extensions.)

I don't know if DNG can contain (optional) spectral response information, but camera makers were traditionally not enthused about sharing such information, or for that matter other information they put in their various raw formats. Nikon famously 'encrypted' some NEF information at one point (which was promptly broken by third party tools).

One problem is that you cannot have a universal format that is both truly raw and doesn't embed camera specific information. Camera sensors from different companies (and different generations) don't have the same color (or if you prefer, spectral) responses with both their Bayer filter layer and the underlying physical sensor. If you have truly raw numbers, you need the specific spectral response information to interpret them; if you don't need spectral response information, you don't actually have truly raw numbers. People very much want raw numbers for various reasons, and also camera companies are not really enthused about disclosing the spectral response characteristics of their sensors (although people obviously reverse engineer them anyway).

This should be fixed for you now. To my surprise, some legitimate versions of Firefox report 'rv:109.0' with a Firefox/<ver> version that is not 109 (along with all of the crawlers that have latched on to 'rv:109.0' as something they put in their User-Agent strings along with random old Firefox versions).

There was a little glitch in the scraping protection where an errant regular expression briefly blocked all Firefox versions. Which is especially bad because I (the blog author) exclusively use Firefox, so I was blocking myself. The management apologizes for the problem (and generally allows much older Firefox versions than Chrome versions, as people seem to still use them on various platforms).

mypy sadly doesn't accept 'NewType('MyType', Any)'; it complains 'error: Argument 2 to NewType(...) must be subclassable (got "Any") [valid-newtype]'. Possibly this is allowed by other Python type checkers. It is accepted at runtime, and I wish mypy allowed Any as a specific exemption to its checks.

(I'm the author of the linked-to article.)

My understanding is that the original SunOS windowing system was deeply connected to the kernel (for instance, windows were devices). The original purpose of the signal (in SunOS) appears to have been telling graphical programs that their window had 'changed' and needed to be repainted (I don't know if this was simply from resizing or if it was any damage), which makes a certain amount of sense for a kernel based windowing system (the kernel is right there and you need some way to notify programs). The use of the signal for telling programs in terminal windows that the window had resized appears to have been distinctly secondary, and only the latter can interact with their window through in-band escape sequences.

(I'm the author of the linked-to article.)

That's mostly explained in Zack Weinberg's Fediverse post/answer: https://hackers.town/@zwol/111684752785283333

The one open thing is why BSD picked 'l' for lstat() to mean symbolic links (unless someone has asked one of the BSD people that). Possibly it's just because 'sstat()' would look weird.

(I'm the author of the linked-to entry.)

A CLA is a legal contract between two parties. An open source license is a general grant of additional permissions under copyright (and sometimes some additional promises regarding things like patents), provided that the person using those permissions (to make and distribute copies) adheres to certain requirements.

(I am the author of the linked-to article.)

As far as I can see, generalized control flow complicates the situation because you need to propagate a variety of signalling results up the tree. Consider a for loop with a switch inside of it, inside of which is an if with a return; if you hit that return, you need to propagate 'return from the function with this result' out through the containing if, switch, and for loop, which means that the Eval() or equivalent for each of them probably all need to know about all of your early-return options (since a 'break' is different from a 'return', especially if you support multi-level breaks).

(I'm the author of the linked-to article.)

One of the problems with domain restrictions on CAs is that real world geography has turned out to not correlate with domain name 'geography' for general use. Organizations located in a country (or the EU or etc) will register their domains all over, instead of nicely restricting themselves to something under a single TLD or a small number of them, where they could use a restricted-scope CA. This mostly leaves you with organizational CAs, such as government ones (for the government's sites), and they seem to not have been too popular in practice.

This was (is) a lack of clarity in my entry. When the system locks up, it only recovers by rebooting through the BIOS; it doesn't resume operation from some suspension. System logs cut off abruptly at the time of the hang, with nothing abnormal even a few seconds before the time and no kernel messages sent out through netconsole (I don't have a serial console available).

(I'm the author of the linked-to entry.)

Apparently the use of an empty read is to probe to see if you get various errors or an immediate 0-size return. This is merely a 'may' requirement in the Single Unix Specification, so I'm not sure how portable it is in practice.

I was going to assert that read significantly predates getchar, but it turns out that both may have existed from V1 Unix onward. However, getchar in V1 through V6 had a different EOF value from our familiar modern one of (-1); they all instead returned a 0 byte to indicate EOF.

(The V7 getc/getchar/etc manpage notes this as a BUG, although it doesn't specifically document what the V1-V6 EOF was. Presumably everyone who this was relevant for already knew. All of this is based on the historical Unix trees available through www.tuhs.org.)

Funny you should mention that:

  ; ls -l <archives path elided>
  -rwxr-xr-x. 1 cks man 65536 Dec  4  1991 ./bin/bin.ultrix-mips/atty

Raw mode by itself is essentially useless for almost all programs; in order to do anything with it, you already need an entire surrounding collection of code to handle editing (even basic things like backspacing) and so on. But line-oriented input remains useful on its own without additional user-level code, again precisely because of the editing support the kernel already provides for you. Since you already need user level code to make any real use of raw mode, the user level code can also set things into (and out of) raw mode as you need it, so line mode remains a sensible default.

If anything things should move the other way, with the kernel providing more readline-style line editing in 'line mode' so that programs need to implement less of it themselves. But this would provoke all sorts of arguments over what line editing the kernel should support and so on, and then someday someone will suggest that the kernel should let you implement your own in-kernel line editing using eBPF (or WASM, if one leans that way), so people could push their preferred choice into the kernel when they log in or start up a shell on a new pty.

(I'm the author of the linked-to entry.)

I think I can boil down an example about why I prefer not to talk about trust by itself as it relates to CAs. Imagine a hypothetical version of Let's Encrypt that has all of its operational excellence and security, but that uses root certificates that are not cross-signed and not included in any browsers. I would argue that this CA is exactly as trust-able as LE is (by hypothesis its procedures and technologies are the same, and we trust LE's), but clearly it is not as useful as a CA as LE is because it is not included in the root set of any browser (which we call 'trusted' and which generally implies that the people behind the browser believe that the CA will not issue certificates improperly).

If we say that this CA is not 'trustworthy' here, what we really mean is 'this CA is not in browser root sets and so the TLS certificates it issues provoke browser warnings'. This is useful in one sense (it is what most people care about), but I prefer to be explicit about what we mean (partly because 'trust' is a loaded term with tangled implications).

My apologies for not being clear about my DigiNotar and StartComm examples. What I intended to point out here is that in neither case were these failures something which one could have discovered in advance by reasonable amounts of checking how trustworthy the CA was. DigiNotar's compromise was (as far as I know) completely unpredictable and appears to have come from a state-level attack; StartComm actively hid itself and took significant work to uncover. Both CAs would have passed most people's checks done beforehand.

If advance warning is nonexistent, there's no point in even trying to check. If advance warning is hidden, you have a cost/benefit tradeoff to make, and I believe that in most cases (ie with low amounts of money involved), it's not good use of scarce and expensive engineer time to try to assess the state of a CA.

(This is especially the case if your company doesn't already have an engineer who is relatively expert in the CA ecosystem and knows where to even start looking. Without that expertise, would something like StartComm's quiet sale to WoSign have raised any alarms even if you discovered it?)

As far as magic bits go, the one thing that self-generated magic bits can't do is insure that a complete first time visitor can't be MITM'd by an active attacker. Whether this matters (or matters more than the scary browser warnings) depends on circumstances.

I disagree. A CA's actual product (what you pay money for) is some magic bits that put a padlock in the browser URL bar or, alternately, allow you to do HTTPS with browsers without your visitors getting scary warnings from the browser (and attackers being able to MITM your visitors). Since it is extremely rare that CA's are de-trusted by browsers, especially on short notice, any CA that can create this good today is basically as good as any other CA. 'Trust' doesn't come into it as such, and to the extent that you must trust your CA, you cannot feasibly verify that it actually is trustworthy and will remain trustworthy, and thus that certificates it has signed for you will remain trusted by browsers for their duration.

(DigiNotar was trustworthy right up until people discovered that it had been compromised. StartComm actively hid that it had been bought by WoSign; it took people some digging to turn this up and verify it. Other CAs have routinely denied that they had problems when they did.)

Even if you could, picking an especially trustworthy CA doesn't add any extra security for your website, because you are actually dependent on the security of all of the CAs out there, any of which can in theory mis-issue a TLS certificate for your site. Sure, there's CAA records, but this only helps if the CA actually follows the rules properly; CAs can fail here at both a technical and a business level. (One CA recently proposed that it should be allowed to ignore CAA records if the domain owner told it to.)

Even attempting to do basic checking of a CA probably doesn't make sense at a business level unless you're buying certificates in bulk. Suppose that you're buying four TLS certificates and the best price you can get is $30 per certificate per year, or $120 in total for a year. How much engineer time does it make sense to spend in order to pick the most 'trustworthy' CA (ie one that is likely to remain trusted by browsers for a year), given that CA de-trusting is an extremely rare event and if it happens, you're only out another $120 to buy certificates from the next CA?

(I'm the author of the linked-to entry and as you can tell, I have opinions on CAs and the TLS ecosystem. One of them is that Certificate Transparency is a really big deal because it removes a lot of the 'trust me' from the whole CA ecosystem.)

The C standard was (and is) written so that it allowed for machines that didn't have simple pointers; a highly relevant example for the time was x86 machines in real or protected mode using segment registers. Even when such environments have accessible linear address spaces under the hood, comparing the absolute address of two pointers requires some extra work (you have to reassemble the absolute address from the segment + offset pair). In the grand tradition of allowing the most efficient implementation possible, the C standard says that implementations don't have to do this work; they can simply compare pointer values directly, provided that they insure that pointers to the same object work (eg use the same segment + offset pair, which they normally naturally will).

The standard was also written to allow for C interpreters, where you may not have an underlying linear memory model at all and all pointers are represented internally in some complex way (for example 'object ID + offset inside object'). Here you don't have any particular idea of 'an address' as a distinct thing and so you can't naturally compare two pointers to different objects in any meaningful way.

For switching to the real root filesystem, you want to read about the pivot_root system call, for example http://man7.org/linux/man-pages/man2/pivot_root.2.html (the usage examples in the pivot_root(8) manpage will help).

When the initramfs PID 1 is something simple like a shell script, I believe it just exec's the real root filesystem /sbin/init after the pivot_root and that init starts from scratch. For Linuxes where systemd is the initramfs init, there's a magic internal protocol to let a running systemd re-exec a new systemd binary while preserving all of its runtime state; this is used, for example, during systemd package upgrades and can be done deliberately with 'systemctl daemon-reexec'.

(I'm the author of the original entry.)

How much things have changed depends on what level you look at the boot process at. For example, before 2.3.41 introduced the pivot_root system call, the starting process in the initial ramdisk wasn't run as PID 1 (as far as I remember and can tell from the remaining kernel code for this). The kernel also used to be far more willing to do things itself, such as assemble software RAID devices; modern Linux pushes all of that into the initial ramdisk user level code, or even later in boot.

At the broad level, though, yes absolutely. Unixes with System V init have been drawing a distinction between 'single user' boot activities like fsck'ing the filesystems and 'multi-user' ones like starting daemons for a long time, so that's a two stage boot. Linux booting became three stage once it added initial ramdisks so that the core kernel didn't have to have to build in all of the pieces necessary to get the root filesystem.

(I'm the author of the linked-to entry.)

In many cases, the IP is the thing that comes first and foremost. You'll put it in DNS, you'll bind services to it (often because that's what the DNS is pointing to) or use it for outgoing traffic, and so on. Any interface name for it is simply an implementation detail, and you can live without it and even simplify your life if you don't have to come up with and keep track of those names.

(This is especially the case if you may at some point shift an IP alias between systems in order to move traffic from one to the other.)

You can enable legacy extensions but this is not the same as being able to use them. Many significant and important-to-me legacy extensions (such as FireGestures) are currently non-functional on Firefox Nightly and have been for some time. I can only assume that Mozilla has decided that they don't care about keeping any legacy extensions working other than their own (they have some).

As it happens, this isn't the case. Sun introduced NFS in SunOS 2, which unsurprisingly had the BSD view of load averages and did count NFS IO as short-term waits. A NFS server with problems could send client load averages up to monstrous levels (a load average of several hundred was not unusual, all of it from processes waiting for the NFS server) and many sysadmins got to see this first hand at the time.