ahl

So while most of the software is open source rather than proprietary, you still have a fair point that customers pay for support (as they do with most enterprise products). One could theoretically use the product without first-party software updates, managing the open source oneself... but that would have practical impediments (and runs counter to the all-in-one simplicity that customers value in the Oxide product).

Two points about your last point. First, software improvements benefit all customers; as the business grows, the effective cost per customer shrinks. Also, most customers grow their Oxide deployment or will replace hardware after a depreciation cycle. The sustainability of investments into the software (and the product generally) is on solid ground.

Oxide uses illumos under the covers That's in no small part due to the composition of the early team--but we have benefitted tremendously from features (ZFS, SMF, FMA, Zones, bhyve, etc.) and observability infrastructure (DTrace, mdb, libproc, etc.) folks on the team have worked on for decades.

This implies that the goal of compensation transparency is to satisfy people's curiosity. I don't think that's the case. Regardless of how it's denominated, I think the goal of compensation transparency is ... uh... equity... no pun intended.

But the real hero was the person who walked 100 yards to home depot and back with the push broom!

What makes you say that? We talk about salary compensation, yes, but there's also equity compensation.

I can't speak for everyone at Oxide, but I'm practically certain that none of us wants to live in Sunnyvale.

Several mistakes here, perhaps most egregious: Sun Microsystems might have made some people rich but that was looooong loooooong ago.

Which is $167,202 in pre-pandemic, inflation-adjusted dollars.

Equity is mentioned in the previous post:

Some will say that we should be talking about equity, not cash compensation. While it’s true that startup equity is important, it’s also true that startup equity doesn’t pay the orthodontist’s bill or get the basement repainted. We believe that every employee should have equity to give them a stake in the company’s future (and that an outsized return for investors should also be an outsized return for employees), but we also believe that the presence of equity can’t be used as an excuse for unsustainably low cash compensation. As for how equity is determined, it really deserves its own in-depth treatment, but in short, equity compensates for risk – and in a startup, risk reduces over time: the first employee takes much more risk than the hundredth.

I didn't watch it in its entirety, but I'm familiar with Allan's work. There doesn't seem to be anything fundamental to the design of ZFS that his talk highlights as limiting, but there are many areas that have benefitted from optimization and tuning.

The claim is Collison directly to them:

Synadia CEO Derek Collison told Runtime Tuesday that the company intends to transfer the NATS trademark to the CNCF at some point in the future "because we just feel that the damage to the ecosystem and the ugliness is not worth it for anyone." While nothing will be official until the lawyers sort it out, Collison said "my hope is that [this dispute] forces some open dialog outside of the CNCF and outside of NATS about the state of open source. I think open source for companies like Synadia is at a crossroads."

Embarrassed to say I had forgotten writing that and LOLed myself… no more reliable audience for your sense of humor than yourself!

Maybe! Bryan and I were pretty close to the case and to the implementation of ZFS. But maybe Apple did detect some smoking gun of which somehow we were unaware. I (still) think Jonathan’s preannouncement was the catalyst for Apple changing direction.

Seems specious. Patents don't preclude one from overtly trying to compete; they protect specific mechanisms. In this case either ZFS didn't use the same mechanisms or the mechanisms themselves were found to have prior art.

It was! And Apple seemed fine with including DTrace under the CDDL. I’m not sure why Apple wanted some additional arrangement but they did.

I don’t recall that being the case. Bonwick had been thinking about ZFS for at least a couple of years. Matt Ahrens joined Sun (with me) in 2001. The Afara acquisition didn’t close until 2002. Niagara certainly was tantalizing but it wasn’t a primary design consideration. As I recall, AMD was head and shoulders above everything else in terms of IO capacity. Sun was never very good (during my tenure there) at coordination or holistic strategy.

Certainly one would build something different starting in 2025 rather than 2001, but do you have specific examples of how ZFS’s design holds it back? I think it has been adapted extremely well for the changing ecosystem.

ZFS was developed in Solaris, and at the time we were mostly selling SPARC systems. That changed rapidly and the biggest commercial push was in the form of the ZFS Storage Appliance that our team (known as Fishworks) built at Sun. Those systems were based on AMD servers that Sun was making at the time such as Thumper [1]. Also in 2016, Ubuntu leaned in to use of ZFS for containers [2]. There was nothing that specific about Solaris that made sense for ZFS, and even less of a connection to the SPARC architecture.

[1] https://www.theregister.com/2005/11/16/sun_thumper/

[2] https://ubuntu.com/blog/zfs-is-the-fs-for-containers-in-ubun...

Back in 2016, Ars Technica picked up this piece from my blog [1] as well as a longer piece reviewing the newly announced APFS [2] [3]. Glad it's still finding an audience!

[1] https://arstechnica.com/gadgets/2016/06/zfs-the-other-new-ap...

[2] https://ahl.dtrace.org/2016/06/19/apfs-part1/

[3] https://arstechnica.com/gadgets/2016/06/a-zfs-developers-ana...

Sometimes data on disk and in memory are randomly corrupted. For a pretty amazing example, check out "bitsquatting"[1]--it's like domain name squatting, but instead of typos, you squat on domains that would bit looked up in the case of random bit flips. These can occur due e.g. to cosmic rays. On-disk, HDDs and SSDs can produce the wrong data. It's uncommon to see actual invalid data rather than have an IO fail on ECC, but it certainly can happen (e.g. due to firmware bugs).

[1] https://en.wikipedia.org/wiki/Bitsquatting

It's very amusing that this kind of legend has persisted! ZFS is notorious for *noticing* when bits flip, something APFS designers claimed was rare given the robustness of Apple hardware.[1][2] What would ZFS on iPhone have looked like? Hard to know, and that certainly wasn't the design center.

Neither here nor there, but DTrace was ported to iPhone--it was shown to me in hushed tones in the back of an auditorium once...

[1] https://arstechnica.com/gadgets/2016/06/a-zfs-developers-ana...

[2] https://ahl.dtrace.org/2016/06/19/apfs-part5/#checksums

Not sure how I fat-fingered Don's LinkedIn, but I'm updating that 9-year-old typo. Agreed that Don is a delight. In the years after this article I got to collaborate more with him, but left Delphix before he joined to work on ZFS.

available here https://share.transistor.fm/s/e2538f7d

Uprobes has—I believe—DTrace USDT-like functionality. This seems interesting and different: sending the tracing facility into the traced/victim process. Whereas DTrace and ePBF typically trap into the kernel to execute and event, this approach allows for potentially much less overhead.

How important is enabled probe effect? Of that I’m less sure.

That's entirely fair: Sun wasn't serious about most of the products they rolled out... and customers noticed.

It’s wild that we built it with 60 people!

It won't. In the same way that AWS customers aren't debugging hypervisor, or Dell customers aren't debugging the BIOS, or Samsung SSD customers aren't debugging the firmware. Products choose where to draw the line between customer-serviceable parts and those that require a support call. In this case, expect Oxide to fix it when something doesn't work right.

Much appreciated!

Do we think it would be possible to replace the real Bryan Cantrill with this? Asking for several friends.

I don’t follow? You don’t think they can set up CI and migrate issues? It seems unlikely that there will be major breaking or architectural changes imminently.