_kp6z

The author also drastically overestimates the capabilities of cloud vendors. CDN and edge have been and will continue to be great business (as you well know) despite pundits predicting their sublimation to cloud subsidiaries for over a decade. For one, capacity planning a public cloud is a lot different and a lot harder than cdn/edge. So the cloud vendor must then act a lot like a cdn/edge, which none have shown any competence in doing. There's a reason Amazon still uses Akamai, Level3, and others for their own Prime Video. CloudFront is a joke in comparison to any prime CDN in terms of network breadth, latency, and overall support from the vendor.

Apple has had the performance per watt crown for quite some time. These chips are not very good for anything outside of prototyping, otherwise hyperscalers would run a sea of small nodes.

illumos concern trolling, this has nothing to do with engineering leadership and is just lashing out a bad faith personal attack because you are friends with joyent people. Being unable to acknowledge that he served his sentence and is reformed indicates you are just hung up on a failed project that is as useful as spilled milk.

Brian Behlendorf and Matt Macy are both fantastic engineers, other people contributed but this is ultimately the kind of thing that must be led by a small number of (in this case two) extremely senior engineers working together for a common outcome with raw technical ability to get everything off the ground and prove it works; and refined leadership skills to stick around til it's actually done!

Google's handling of these critical archives they were given is pretty abhorrent. The usenet archives should really be made public since there is no business value to them and they don't care about usenet.

I have a huge soft spot for NetBSD because it was my first BSD and first foray into kernel work. The more alarming issue affecting Open and Net is that commodity computer architecture has wildly changed in the past 10 years. A late 1980s style locks and CVs approach to SMP is only good for a handful of cores. Consequently, if you run one of these on what are becoming standard desktop class systems like Ryzen, you are throwing away a great deal of performance (and therefore energy efficiency). Basically modern computers look a lot like an HP SuperDome or whatever from 2000 which is wild and inconvenient for programmers, but reality. FreeBSD (HEAD) is now generally competent on something like an Epyc or multi-socket intel box, and there isn't much left to eek out on an 16-thread laptop.

I funded and helped stabilize a technology called Epoch Reclamation into FreeBSD that can be thought of like a cleaner version of Linux' RCU at least for tracking object liveness. One of the most interesting things I've seen go into FreeBSD recently is a further improvement on that, a novel form of safe memory reclamation directly in UMA: https://reviews.freebsd.org/D22586. This is really exciting because for instance RCU suffers from time of use to time to free issues in Linux, and this new holistic approach really does not for fast turnover workloads.

NetBSD does have a form of SMR called pserialize(9), but ignoring some of the algorithmic weaknesses it also has a lot of global or oversized locks in critical subsystems like the VFS and network stack that turn it into a fairly non-SMP friendly system let alone consideration for different memory latencies like NUMA.

A lot of these ideas from FreeBSD can be copied into Net and Open but the code bases have diverged greatly where it's not some simple forklift (for instance Net and Open use a similar kernel memory allocation that is totally different than Free).

Sounds like a pretty clueless layoff, I guess I expected better from Mozilla than usual corporate derp. If there was truly no dead weight, surely the management could have scaled back their own comp for misdirecting the company? Very few people understand what it means to be a leader in corporate world.

Heh except if you actually read the comments I present Linux as the high water mark twice for security and stability.

Maybe - and I'm only saying this because I care - you should consider through hiking and come back to the discussion when you are less easily offended by the facts at hand into low grade trolling and have requisite attention span and reading comprehension to participate meaningfully in the conversation.

I don't think I can agree scientifically. OpenBSD developers do subjectively seem to think more holistically about security, and having less features lowers the total number but not the ratio and in practice it doesn't seem to have what I would consider a statistical magnitude difference in the type and ratio of incidents that plague C Unix-likes. In the article the author links to one of Ilja van Sprundel's excellent talks; the takeaway is basically that more eyes actually work for security and you could therefore infer Linux is a magnitude more secure than BSD.

I've been quite negative in this thread because the overall debate on these topics is pretty frivolous. We already have the tools at our disposal to do magnitude greater secure software and splitting hairs over which BSD has better defaults is missing the forest for the trees.

I use OpenBSD for fun on a couple systems and it is a cute OS, there are things to love about it. But the grandstanding is funny.

Yes, run https://github.com/antonblanchard/will-it-scale on operating systems with hilarious grandstanders like OpenBSD and Illumos, then run it on Linux and compare the results. Run a highly parallel make on a modern server and tell me which OS remains usable for other work. OpenBSD runs acceptably on 2-4 core laptops with HT disabled because a desktop workload doesn't typically invoke a lot of the kernel implementation during steady state, but you can do it there with a moderate disk and network load. That said, syscalls are more expensive in OpenBSD than contemporary operating systems so even as a uniprocessor OS it is funny.

Sure, say you configure a Linux, OpenBSD, and FreeBSD machine to provide some useful service. A "stack" as they call it these days. None of these OSes are going to have a large attack surface outside that "stack", and that is the primary liability in running an internet service. There are relatively rare occurrences like the TCP SACK bug that affected Linux and Netflix' non-default FreeBSD RACK TCP stack. Every UNIX-like written in C without formal verification is subject to incidents like that from time to time. Every other security concern is addressed in an OS-agnostic fashion with failure domains, limits of scope/networking, access controls, monitoring, and operations procedures.

Now what Linux, and to a lesser degree FreeBSD, can do is survive real world usage at scale. OpenBSD cannot. It would fall over or require magnitude more machine count to do the same workloads that people use Linux and FreeBSD to run. So all the exploit mitigation diatribe and "great defaults" and pet projects are cute but funny when you try and throw shade onto others with them. As I said, there are some kernels of truth that pertain less about security and more about overall health in FreeBSD this doc accidentally hits on, but running some hobby software like opensmtpd on openbsd isn't going to save the world from real cybersecurity issues.

A more plausible explanation is that the xen networking path is simply expensive, the intel VFs are limited by queue count and silicon (i40e isn't a great ASIC), and the Annapurna part is really an ARM64 NPU. NPUs have been abandoned by most silicon vendors and have a tragic history. It's simply hard to make NPUs work right at attractive price/power/performance and at high speed versus fixed function scatter/gather I/O units coupled with general purpose CPUs running software network stacks. The only benefit Annapurna gives EC2 over a software device model is a hard security boundary of effectively another computer inside the computer for Nitro metal as a service. I think this is one reason why EC2 is limited to 25G while 100G has been commodity for a long time.

Here is a demonstration of a software stack that can scale toward hardware limits without relying on a particular vendor https://www.slideshare.net/SeanChittenden/freebsd-vpc-introd.... This approaches 100G line rate for large packets which is what it was optimized for. I don't know PPS at low packet size but do know what would be required to optimize that use case and it could be done pretty quickly.

It's on a quad core machine with a tiny heap to manage, even OSes like OpenBSD do alright with Giant kernel locking at that level of concurrency.

This would turn into a massive research project on GC and language runtimes where the commodity server market already is and especially where it is going considering a dual socket AMD Rome will have 256 threads and a single socket POWER10 would have the same. To consider that project in terms of time scale and cost, one can look at the history of JVMs. Go's type system might make some things slightly easier but there is no implicit scalability codex they've unlocked beyond state of the art JVMs that have been evolving for 15 or more years.

This is why Rust brings something novel to the table for systems programming. With the borrow checker, memory safety can be added in many cases without runtime overhead and follow on contention.

An extended though, we need to be moving _toward_ sympathy with the on chip network and implicit message passing nature of cache coherency policy. This is mostly a library/ecosystem shift rather than language.

I'm no fan of php, but you realize the home routers you recommended earlier typically have CGI UIs written in _C_ by people that don't really understand strings and pointers?

I think this is some kind of psychological projection/rationalization because the statement "Having been well and truly spoiled by illumos threading (and especially illumos kernel threads)" is actually quite laughable.

It may be equal parts GPL avoidance. Broadcom switch ASIC PDKs are a kind of hybrid kernel and userland application with no legacy reason, so I assume it is just arbitrarily about working around a restrictive license.

Going back to the intel FUD paper, if you consider volume shipment as success as in their document, uboot looking thing with FDTs are the most successful paradigm by orders of magnitude.

The kind of systems using FDTs aren't going to switch I don't think. I have to imagine that ARM64 UEFI was done for marketing and by clueless product management. The CPUs and integration of ARM servers have been so pathetic, nobody was buying, and they were fumbling around at this level while ignoring the elephant in the room. TX2 and Centriq are the first realistic implementations for general purpose servers, and now they are unfortunately saddled with UEFI and ACPI.

This was a rare 2-3 decades kind of mistake. Not a lot of software gets that privilege.

That is almost entirely FUD, every Mac before the intel switch had OpenFirmware. FCode was elegant in that it is cross platform, and the option ROMs can work across multiple uarchs. The only valid point is that Forth is obscure. I don't know how much that matters in reality due to how obscure working at that level is itself, as an OS dev you're working on the device tree and runtime services in a language like C. But I will grant benefit of the doubt since I'm an OS dev and not an option ROM dev.. and say, well, Petitboot is the logical endgame as the industry has made Linux uber alles and you're going to be writing drivers for it anyway.

UEFI is a great example of second system syndrome. In the BIOS world, more featured boot loaders were chained in early (think pxelinux, grub, *BSD loaders etc) so even though it was primitive it wasn't the typical UX. UEFI is almost trollingly bad, the worst amalgamation of low level firmware, boot loader UX, and OS->firmware runtime services I've seen when contrasted to OpenFirmware, uBoot, petitboot.

OpenFirmware was a much more elegant technology sitting around for the lifetime of modern x86 but intel had to be different.

I like the direction IBM is going with OpenPOWER.. petitboot/kexec by default https://sthbrx.github.io/blog/2016/05/13/tell-me-about-petit... although all the firmware sources are on github so you could do whatever the heck you want. It makes intel look positively oppressive.

One other point, these cloud companies like Amazon and Google do a great deal of self congratulations for some pretty embarrassing architectures. Necessitating million server datacenters is as WTF to me as being proud of millions of lines of code when someone else is doing it with magnitudes less. 40G networking has been commercially viable since the top of the decade and 100G (or partial variants) for a couple years now while ENA tops out at 20G. They are basically asleep at the wheel because their architectures are predicated on poor assumptions, shirking understanding of computer architecture for 10s of thousands of SREs and "full stack developers".

Dovetailing a bit, back in the commercial UNIX and mainframe markets it is pretty common to have 80%-90% percent system utilization. In the Linux world, it's usually single digits. For some reasons (I guess it's more inviting to think holistically due to the base OS model), we are getting those 90% figures in the BSD community. See drewg's comment, WhatsApp, Isilon, LLNW:

I led the creation of an OS team, to look from first principles, where we were spending CPU/bus/IO bandwidth, and focusing on north-south delivery instead of horizontal scale. A team of 5, we are able to deliver significant shareholder value http://investors.limelight.com/file/Index?KeyFile=38751761.

As a long time user of OpenRC, I was pretty sad when Debian jumped into systemd by default instead of sticking to something like OpenRC as the default. It's light weight, easy to use, embraces POSIX, and a good modular building block for both server operating systems and not over reaching on the plumbing desktop users want.

Cheers to TrueOS which is done by an astonishingly small dev team and part time at that as they are FreeNAS devs. If you ignore the hit and miss graphics story shared with FreeBSD for now, TrueOS has some of the best defaults and out of the box experience I've seen.

I wonder if there is any professional psychology on this kind of thing (link me!) but my layman observation is that most organizations tend toward median smoothing at best or lowest common denominator at worst for performance. That is of course not how you achieve high performance. But high performance scares lower performance, the conflict is bidirectional. From the article it's pretty obvious he was a high performer. High performance means having opinions. Opinions are contestable, and create conflict. Conflict is part of a high performance organization. Lower performers are terrified of conflict with higher performers. Therefore median and lowest common denominator organizations force high performers out. It's kind of ridiculous to think about that happening in academia, but today I learned!

When you switch to the arts and sports we LOVE to see extreme outliers and go to great lengths to assemble and prop them up. It's visceral excitement to see 320lb freaks of nature face off on a line of scrimmage in an NFL game. It would be cruel to have them face off against a highschool team or even just mix them in on the same line. Yet that kind of disparity happens all the time in other types of human assembly.

So I look at this from the other side. He did his deed at the University, made a mark, and got rejected for having high impact good, neutral, or bad. Some people were uncomfortable with high performance for reasons which could range from pettiness to valid criticism.

What you would call lack of tact I might call integrity. I am sure he will land somewhere else and do great things. Industry is more fluid and amenable to assembling high performance, especially in startups.

Yes, I use it for my personal web servers. I completely rid myself of Linux when Debian switched to systemd. I run nginx for static and proxy services. I wrote some Scala/Play apps that I run in jails on OpenJDK8. I run some open source PHP, Python, Ruby and C and perl CGI scripts for various admin things and "cloud" things but all on my own hardware. I use iocage to manage the jails. I replicate the jails between two physical servers asynchronously, and also send them to my house for an off site. I run PostgreSQL on the two servers in a failover setup with WAL shipping. It takes about 30 minutes a month to keep security patched. I use saltstack to manage as much as possible. It took about 2-3 weeks to set everything up using just evenings and weekends, it's a lot more hacky than stuff I do for dayjob because I'm the only one that needs to understand the setup so I can elide some robustness I'd use in a team environment (i.e. no test suite, no branches, no CMDB just simple text files assigning roles, no multi-platform or abstractions everything is decisivly coded for my tech stack of choice like FreeBSD, Postfix, Dovecot, PostgreSQL)

I also use it at work. The scale is much different as one of the largest CDNs, the network interface stats are eye popping. This is proprietary caching software and nginx and apache for certain types of origins or admin services.

I did the opposite and can't believe how much better my life had gotten because my iPhone is just a simple tool that I use for communications and don't think about it as a project. With Android, I always wanted to tweak silly things and run Cyanogenmod because the handset firmware was always so bad and vulnerable. On several occasions I'd bricked my phone requiring hours of recovery, or had transient failures of cell service and communications issues. I guess if you have the right level of discipline, apathy, or use a Nexus device that may be more Apples to Apples (harhar).

Howard Chu of LMDB fame really bags on LSM:

https://twitter.com/hyc_symas/status/650799297143836672

https://twitter.com/hyc_symas/status/657399873562656768

I'm not a member but one thing that's kind of interesting is having a rootless, shell-based community. We have this at work on our jumphosts and it's generally a Good Thing. Sharing datasets and scripts is just a filesystem namespace operation. It's up to the system operators to keep the machine running well and influencing how it is used by i.e. setting up certain packages and services. It's incredibly productive to be in a tmux session with vim, mutt, tin, man pages, ssh, etc as your interface and minimal distraction.

If you think strace is useful, wait until you try dtrace.