Rust, Wright's Law, and the Future of Low-Latency Systems

https://www.youtube.com/watch?v=cuvp-e4ztC0

by zdw • 5 years ago

120 76 5 years ago

scrubs2• 5 years ago

Ummm ... you cannot reason from H/W (video spends 80% of its time recounting H/W manufacturing history which is in the domain of natural sciences with a formal model e.g. physics, chem, statistical physics) to software. The first 80% of the video is not relevant.

replygirl• 5 years ago

sorry but the kantian project is dead bro.

scrubs2• 5 years ago

OK, downvoted. The point of my comment was not that software cannot be aligned or co-located with hardware. As others pointed out C/C++ is just as capable. My point was that spending the first 75% of the video recounting H/W advances which is based in formal science (chem-eng and the rest) is orthogonal to software. Moore's law does not carry over into software. The video could have just started by pointing out with H/W specialization there are more places to put Rust code. This way 99% of the video could have been on the OP's like of Rust. Right? The top comment shares much the same sentiment: "Skip towards the end and Cantrill..."

ozten• 5 years ago

These hidden cores are fascinating.

In the past, I was interviewing at a hardware company and one of the things they had just discovered was an "unused" and undocumented processing unit that they could run a process on to get squeeze out more compute from the cameras they used for calibration for the hardware bill of materials that they had already settled on.

bcantrill5• 5 years ago

I know that I recommended it in the talk, but I highly recommend Timothy Roscoe's OSDI keynote[0] -- and may also be worth catching the Twitter Space we did a few weeks ago discussing it.[1]

[0] https://www.youtube.com/watch?v=36myc8wQhLo

[1] https://github.com/oxidecomputer/twitter-spaces/blob/master/...

RobLach• 5 years ago

That keynote is great. It feels like looking up and seeing the sun from the floor of a deep hole you've been too focused on digging.

CalChris• 5 years ago

I liked your talk but I didn't get the connection with Wright's Law. You did a good presentation of WL but I just didn't get its connection with low latency and future hardware software co-design.

zja• 5 years ago

I discovered Oxide's youtube channel a couple weeks ago, and I've been really enjoying listening to your twitter space discussions. Thanks for all the interesting content!

aidenn02• 5 years ago

Given that you continuously recommend Roscoe's keynote, and you are working on a message-passing microkernel; will that microkernel support message passing between heterogenous cores with heterogenous physical address spaces?

dralley• 5 years ago

Hey Bryan, just curious, what are your thoughts on the tradeoffs that Zig is making compared to Rust?

devmunchies• 5 years ago

he says that rust can fit into these hidden cores (special compute elements) but that there won't be dedicated CPUs or memory.

I'm not a hardware guy so I don't have the imagination of what you could do on these cores without CPUs or meaningful memory.

ncmncm20• 5 years ago

They have CPUs, and memory. The memory is generally managed in a more restricted and idiosyncratic way than POSIX programs like to use. The CPUs are often not what you are used to compiling code for.

The tricky part is providing a programming model that is usable, understandable, and useful, given the wide variability of the conditions the code would need to operate in. The eBPF project has made remarkable progress in that direction. eBPF code can be generated from C, C++, Rust, and a bunch of other languages. The "no_std" feature in Rust has no substantial role in getting your code compiled down to eBPF, in preparation to be further translated to the object code that actually runs in the peripheral gadget.

wyldfire• 5 years ago

Skip towards the end and Cantrill talks about:

* no_std

* composability of no_std crates

IMO the development-time packaging (cargo) means that as a developer I can have a deep and broad list of dependencies without having to orchestrate my development environment. This is Rust's killer feature, I agree. The fact that you can do it w/no_std is also very awesome.

Does anyone have more info about "Hubris" - the OS he refers to as under development?

zozbot2344• 5 years ago

IMHO we need to allow for local allocators as well as a bunch of other stuff (in-place constructors for pinned objects, ala C++) before we can claim to have true composability in a no_std environment. You see this stuff crop up all the time in "Rust kernel modules" discussions too, and there's a reason for that.

pas2• 5 years ago

For anyone who wants to know a bit more about this, this a recent talk about how to implement move constructors in Rust: https://www.youtube.com/watch?v=UrDhMWISR3w

steveklabnik3• 5 years ago

We will reveal more about hubris at the conference he referenced at the end, which is happening end of November/start of December.

I am extremely excited.

PeterCorless• 5 years ago

We also had a great chat about Hubris in the Speaker's Lounge at the event. Brian was definitely on fire about the topic!

The one thing I wanted to point out is that there's already a "namespace collision" when you search on "Rust" and "Hubris" — a language called Hubris written in Rust:

https://github.com/hubris-lang/hubris

filereaper• 5 years ago

Really looking forward to Hubris and especially the design decisons made in building it. Like Bryan says, the why behind it all.

Cheers.

mooman219• 5 years ago

I agree that no_std is incredible. I really want to see more crates embrace it and encapsulate their no_std logic away from their standard logic. I very often see crates that are like 95% of the way to no_std but then choose to bundle some standard only features without flagging them.

I wrote fontdue [0] (which is very incomplete spec wise) because there just wasn't another font library that was no_std at that time. It felt like the existing libraries were in an arms race for gpu caches and bundling file loading. Like if I wanted to commit to a running on a platform I'd do the sane thing and use harfbuzz or the platform APIs.

It's very naive because I don't understand all of the complexity, but I'd really like to see the standard library being easier to piecewise implement from crates. Like a standard trait library and a standard implementation library for those traits.

[0] https://github.com/mooman219/fontdue

pas2• 5 years ago

Like a standard trait library and a standard implementation library for those traits.

This sounds like a very good idea. (The thinness of the std future API paved the way for multiple async runtimes. And sowed confusion! evil laugh But seriously, having a high throughput optimized runtime like Tokio and a size optimized like smol is a good thing.) But a very much non trivial amount of work. Has this been discussed on internals?

ncmncm20• 5 years ago

The first 15 minutes are just historical rehash, and can be skipped over without missing anything at all.

The rest just says that compute elements are showing up in our peripherals, and that we will need, and want, to program them. (People have said this for decades, but there has been little movement toward enabling it, because it will always be hard, for reasons.) Then it makes the absurd claim that Rust is uniquely capable of programming those elements, because it has a feature called "no_std", where your build fails if your code tries to use any standard library features.

Of course, all those peripheral processors are already programmed, today, almost all in C and C++. It is arbitrarily hard to run your own code on most, today, because where it is possible at all, you need to solder in a JTAG connector and reverse-engineer the code in there to figure out how your code can operate in the environment.

Lots of peripherals, though, get their object code loaded into them by the kernel driver at startup before they will do anything useful.

Maybe someday it will get easier to download your own code into them from a running system, to extend what they are already doing, and maybe someday somebody will document what your code would need to do to contribute to such operation. But if that ever happens -- and, to be clear, it is happening in certain, select places, like some NICs -- Rust will have no advantage over other languages. Using "no_std" will not materially help.

For example, a company Netronome has today a NIC that lets you run your own code on it, e.g. to filter or alter packets before they get DMA'd out to the ring buffer where the kernel sees them. It is programmed using eBPF, which is a virtual object code format that a kernel driver will translate to native machine code. eBPF code can be generated by running LLVM over a file generated by a compiler for C, C++, Rust, or really almost any language that can be compiled to LLVM intermediate code, and offers some way to call out to a C library API.

The Netronome kernel driver takes your eBPF object code, compiled from any language, translates it to machine code for one of the cores in the NIC, copies that onto the NIC, and patches it into what is already running there.

Of course none of the other languages have "no_std", and don't need it. Your Rust code doesn't need it either. Don't want to use library features? Just don't use them. The things an eBPF program is allowed to do are quite limited, but surprisingly powerful, including calling out to a special ("standard") eBPF library. That Rust's sum types are core language features gives it no advantage over the equivalent C++ Standard Library features that (also) do not depend on linking to a runtime support library.

secondcoming• 5 years ago

The first 15 minutes are just historical rehash, and can be skipped over without missing anything at all.

I think the software world has got to the point where the obligatory Moore's Law graphs, etc are no longer required when talking about CPU performance.

brundolf8• 5 years ago

I thought the "historical rehash" was interesting and well-presented

gnurizen• 5 years ago

Anybody else chuckling at the irony here that eBPF is inspired by dtrace which was invented by Cantrill?

ncmncm20• 5 years ago

No.

If you want to do stuff like this, anything you use to do it will have to look a lot like eBPF. eBPF doesn't make it easy, it only makes it possible. But dtrace was not eBPF.

zozbot2344• 5 years ago

You can't just take Rust code and transpile it into eBPF, it's not a Turing-complete language. Now WASM with the addition of some tailored API's could do what you're talking about.

ncmncm20• 5 years ago

Yet, Turing-complete or no, Rust (like C, like C++) can in fact be compiled to LLVM intermediate code, and that intermediate code, provided it conforms to eBPF requirements, can in fact be compiled down to eBPF. And, that is how what I'm "talking about" is in fact done today.

It is not being done with WASM, and probably will not be.

Animats• 5 years ago

The first 15 minutes are just historical rehash, and can be skipped over without missing anything at all.

Right. Short version: Moore's law over, fabs too expensive.

Then it goes on to make the absurd claim that Rust is uniquely capable of programming those elements. This is asserted to be because of a feature called "no_std", where your build fails if your code tries to use any standard library features.

He's a bit vague there. What he's getting at, though, is that a common heap is a problem when you have enough CPUs. Shared-memory multiprocessor architecture is hitting a wall. Caches, and cache sharing, and cache intercommunication have made it possible to get a large number of CPUs to pretend they share memory. But they really don't share, and trying to maintain that illusion adds considerable overhead.

This is an old observation. It's led to lots of distributed systems - the Transputer, the Cell, and a whole bunch of experimental one-offs. All failed.

Now, there are successes of loosely coupled parallelism. GPUs. Neural net simulation chips. Bitcoin miners. Supercomputers doing finite element analysis. Those are useful for very specific problems where you need a large number of semi special purpose compute elements. What this guy may be thinking is some general product to do all that.

But he doesn't say much about how to do that.

ncmncm20• 5 years ago

He makes the same mistake that people always have in promoting what the ISO C and C++ Standards call a "free-standing implementation". That is supposed to be a build mode intended for use in embedded systems, where the program is itself the whole system, so it cannot rely on OS services.

This gets conflated, absurdly, with an inability to allocate, manage, and use heap memory. Billions of embedded devices reserve and manage heap memory without difficulty. Indeed, every OS kernel is running in just such an embedded environment. Managing memory for its own use and on behalf of user processes is among its chief activities.

The problem with defining a "free-standing" version of a Standard is that, in practice, real systems invariably need to use some of what is not specified to be part of the negotiated "free-standing" subset. There are at all times active proposals to add this or that extra library feature to the "free-standing" subset. Meanwhile, the language implementers have very little incentive to package any "free-standing" subset at all, because no one such subset that any substantial number of embedded users can actually use is possible.

In practice, embedded-system developers use the regular toolchain, and just link a runtime support library that implements the things they need. So, for example, C++'s std::vector will, by default, call operator new(). But any particular use of it, in an embedded program, may specify a custom allocator, and the object code for the program then ends up with no calls to operator new(), and so links happily to a runtime library without one.

brundolf8• 5 years ago

Of course none of the other languages have "no_std", and don't need it. Your Rust code doesn't need it either. Don't want to use library features? Just don't use them.

1) no_std allows composability. One of Rust's biggest strengths is its package ecosystem, and I imagine this will only become more true in the increasingly "weird" and sprawling hardware ecosystem described by the OP, where the ability to re-use code that other people might have written for your hardware could save you from having to "solder in a JTAG connector and reverse-engineer the code in there to figure out how your code can operate in the environment". "Just don't use [the standard library]" doesn't work when you want to use third-party libraries.

2) no_std's static checks are nice even in your own code. The value of static analysis has been argued about so many times in threads like these that I don't feel like re-treading that ground. But suffice to say: it's clear some people think they can write flawless code and don't see the benefits of assistance, but plenty of others know their limitations and benefit from static analysis.

3) Rust seems to have straddled an interesting and useful line in terms of which features are available in these limited contexts and which ones aren't, giving you as much to work with as possible. And thanks to #1 and #2 you can use those features fearlessly; you never have to guess about whether or not they're in the "safe" category. Hygenic macros in particular can act as a compile-time force-multiplier when you want better abstractions but don't want to (or can't afford to) do extra things at runtime.

As always comes up in these discussions: yes, you can technically cover this usecase with C and C++. You can technically cover it with raw assembly code, or binary strings, or a pair of wires. Don't discount ease-of-use as a factor that can take things from "technically possible but practically infeasible" to "doable and worth doing".

ncmncm20• 5 years ago

"you can technically cover this usecase"

Not just technically: essentially all code covering "this usecase" -- operating peripheral cores -- is today C++ or C code.

It would be more meaningful to say you can technically do it in Rust, because it is possible in principle to do it, even though practically nobody ever has, and vanishingly few ever will.

Everyone who makes up an "embedded subset" congratulates themselves for identifying just the right subset. Then they immediately start moving other stuff over the line. And never stop. This is all familiar ground. All that is new is new people noticing it.

An embedded program in any language that relies on a library function not implemented in its runtime environment will, in any case, fail to link. That is static checking. Rust is not doing more static checking here. It is just doing it sooner, and forbidding a truly enormous amount of what would also be useful to embedded programs. Most of us would rather have access to all of that, and let the linker identify what support is needed for what is used.

jmull2• 5 years ago

I'm not sure I'm understanding this correctly... Rust libraries are generally composable, so that's not what makes no_std interesting.

I guess the advantage is that it sets a bar that means "can run in a wide range of environments" and it's a standard bar so that everything that's "no_std" can (more or less) run in the same wide range of environments.

But I don't know. A standard bar is nice, but I'm not sure it's really that much of a help.

I just dabble with embedded programming as a hobby, but it doesn't seem to me like non-Rust approaches (I'm thinking of C and Python) are suffering from a lack of composability.

brundolf8• 5 years ago

So the idea is that the low-level abstraction becomes more like a distributed system where different pieces of actual hardware coordinate independently, instead of where they all go through this single, central CPU that has the final word on everything? Almost like "edge computing" at the scale of the motherboard

bitwize• 5 years ago

I hate to be the smug Amiga weenie but the Amiga was doing this in 1985... the idea that everything goes through the CPU is largely an accident of PC dominance and the dearth of beauty and elegance in system design resulting therefrom.

brundolf8• 5 years ago

Interesting! I didn't know that

zozbot2344• 5 years ago

The early home computers were invariably designed around the video signal clock, so they were quite deterministic and predictable - very little was truly 'independent'. The PC with its custom extension cards was more of a pure distributed platform.

chubot• 5 years ago

Reminds me of Your computer is already a distributed system. Why isn’t your OS? from HotOS 2009:

https://www.usenix.org/legacy/events/hotos09/tech/full_paper...

We argue that a new OS for a multicore machine should be designed ground-up as a distributed system, using concepts from that field. Modern hardware resembles a networked system even more than past large multiprocessors: in addition to familiar latency effects, it exhibits node heterogeneity and dynamic membership changes

And now that I look at the author list, I see Roscoe, who gives the 2021 keynote that Cantrill recommends:

https://www.youtube.com/watch?v=36myc8wQhLo

thelazydogsback• 5 years ago

QNX? - been around for a while - the first OS to run in protected 386 mode FWIR. (And I remember it booting from a single floppy :))

Rust, Wright's Law, and the Future of Low-Latency Systems

Related Stories