RISC-V OS using Rust: Filesystems

http://osblog.stephenmarz.com/ch10.html

by pavehawk2007 • 6 years ago

242 72 6 years ago

osblog.stephenmarz.com

Excellent series of posts. I've been following along with no prior Rust or OS programming knowledge and have found it immensely refreshing.

pavehawk200713• 6 years ago

Thanks! Glad it could serve some purpose!

marricks• 6 years ago

This looks awesome! Yesterday I just started another OS rust walk through and now I have a hard choice between that and this.

Any thoughts on this[1] walkthrough in comparison? This one looks more explicitly up to date.

[1] https://os.phil-opp.com/

pavehawk200713• 6 years ago

I think you'll get something different for each. The purpose of mine is to explore the Rust programming language, but more specifically on the RISC-V architecture. Rust "recently" became out-of-the-box for RISC-V, which has opened up many more avenues for exploration.

As a note, I believe os.phil-opp.com is using the AMD/Intel architecture.

jimbob45• 6 years ago

What do you mean by out of the box for RISC V?

jeffdavis2• 6 years ago

Is the asm!() macro going to stabilize? It seems important to ever make a real OS, right?

pavehawk200713• 6 years ago

Hello, all signs point to no. I've seen many threads about stabilizing asm, but most seem to say no this won't happen. In fact, the Rust compiler now throws a warning that asm!() is deprecated in favor of llvm_asm!().

This is the only reason I am still using nightly builds...because I'm forced to use the asm language feature, which is unstable, and because it's unstable, it cannot be used on the stable channel.

Edit: to respond to the second half, for out-of-the-box Rust builds, yes, I need global_asm! and asm!. However, asm! is for inline assembly only. Originally, I used GNU's assembler to assemble and then linked using GNU's ld. However, I have since switched to trying to make Rust do everything through its build manager, cargo.

jcranmer• 6 years ago

There is a proposal for a new asm! macro: https://github.com/Amanieu/rfcs/blob/inline-asm/text/0000-in... (note that the last updated date was 2 days ago, as of the time of this comment).

It looks like what you're referring to is the interim plan to rename asm! to llvm_asm! (which will never be stabilized), since it's a very thin wrapper over the current LLVM inline asm support, and then replace asm! with a better implementation.

There is a working group specifically for inline asm, so it will happen eventually... but it's not going to be a quick process--think about how long async/await takes, and inline asm is actually rather close to that in terms of all the issues that come out of it.

steveklabnik6• 6 years ago

It's closer than it's ever been! The rough plan:

* move asm! to llvm_asm. https://github.com/rust-lang/rust/pull/68404

* implement the actual asm! we want to stabilize: https://github.com/rust-lang/rfcs/pull/2873

* stabilize it

pavehawk200713• 6 years ago

I stand corrected! Thanks for the info!

jeffdavis2• 6 years ago

Cool!

Curious why not WASM also? I have no use case for that, just asking.

pjmlp2• 6 years ago

Not really, one can always make use of an external assembler.

loeg2• 6 years ago

The minix fs seems to be basically the classic Unix FS design, like UFS/FFS or Ext2, prior to the concept of cylinder groups (at least, per this description and Wikipedia's) and without some of the more advanced features of those filesystems (softupdates, directory trees, etc).

yjftsjthsd-h• 6 years ago

I mean, that's what you'd expect from an OS that was originally for teaching, right? A worked example of the fundamental parts common to most other unix filesystems, just as I would naively expect the minix scheduler to just be round-robin or something, etc.

loeg2• 6 years ago

Well, that was especially true for Minix 1 and 2. Minix 3 is purportedly a production operating system, and ships in every Intel CPU for the last decade or so, right? So while admittedly it may not have significant filesystem needs, it is not purely a teaching operating system anymore.

Nowadays I might point at MIT's xv6, which implements a very similar simple unix filesystem: https://github.com/mit-pdos/xv6-public/blob/master/fs.c

a13692099935• 6 years ago

A couple of minor, but very irritating errors:

The singly-indirect pointer can address 1,024 * 1,024 / 4 = 1,024 * 256 = 262 KiB of data.

256 kilobytes, not 262 'kibi'bytes.

1,024 * 256 * 256 = 67 Mi
1,024 * 256 * 256 * 256 = 17 Gi

64 and 16 respectively.

pavehawk200713• 6 years ago

If that irritates you, please don't check the grammar and errant placement of commas!

All jokes aside. Fixed, thanks.

a13692099935• 6 years ago

Actually, the grammar seems fine? Or at least I haven't noticed any problems with it. TFA as a whole seems pretty good actually. The file sizes are still incorrect, though; it calls the units kilobytes now, but it still claims there's 262 of them.

The seven blocks at the start would make it 263 KB, but I don't think that what you meant.

jfkebwjsbx• 6 years ago

It is actually 256 KiB, not KB.

fluffything4• 6 years ago

This is awesome. I've been following along for a while, so please keep it up.

I wish someday we'll get something like the ESP32 boards but with RISCV.

pavehawk200713• 6 years ago

Thanks!

You might want to check out the Kendryte K210. I'm trying out these boards for development. They are designed with two (I think) 64-bit RISC-V cores with a hardware FPU. https://www.seeedstudio.com/blog/2019/09/12/get-started-with...

I've flashed mine a few times, but I'm still working to getting it to work. The documentation isn't very good, so I'm working to reverse engineer some aspects of it.

This is a low cost alternative. Of course, we could go with the SiFive Unleashed (64-bit) or the SiFive HiFive1, which uses a 32-bit RISC-V CPU in an Arduino form factor. https://www.sifive.com/boards

The SiFive Unleashed cost me a cool $1k, so I'm still scared to mess with it.

FullyFunctional2• 6 years ago

The Kendryte is pretty limited in many ways, not least that it's permanently limited to 8 MiB of DRAM (AFAIK).

The most interesting route is using an FPGA. There are many examples of running RISC-V softcores on FPGAs (for example, Berkeley's RocketChip) and many boards that support a decent amount of memory (even beyond 1 GiB).

MrBuddyCasino2• 6 years ago

No need to wait, the ESP32-S2 has a RISC-V as its low-power core.

sitkack3• 6 years ago

ESP32-S2 has two ULP co-processors, with one based on RISCV instruction set architecture (ULP-RISCV) and the other on finite state machine (ULP-FSM).

ULP-RISCV has the following features:

    support for IMC instruction set
    thirty-two 32-bit general-purpose registers
    32-bit multiplier and divider
    support for interrupts
    boot by the CPU, its dedicated timer, or RTC GPIO

https://www.espressif.com/sites/default/files/documentation/...

colejohnson66• 6 years ago

Have you checked out SiFive’s HiFive1?[0] It’s an Arduino Uno sized board, but instead of a weak ATMega328p, it’s a (relatively) powerful RISC-V processor. It also has a BT and Wi-Fi chip.

[0] https://www.sifive.com/boards/hifive1-rev-b

sl1ck731• 6 years ago

Is there a significant limitation in using Rust on the stable branch? Every interesting tutorial, book, or tool I find seems to use nightly. I know it shouldn't be a big deal but it "feels dirty" and perhaps I should just view nightly as any other language's "stable".

steveklabnik6• 6 years ago

For operating systems development, inline assembly is convenient, which is not yet stable. See the conversation below.

The vast majority of Rust users use stable. You just happen to see a lot of folks who are interested in the cutting edge blog about things. (You may also run into older things where that thing is stable now, too, to be clear.)

gpm• 6 years ago

perhaps I should just view nightly as any other language's "stable".

It depends on the features you enable. Stable is basically just a pinned nightly version with no features enabled, and I'd definitely view "nightly with no features" as more stable than most languages.

Using some of the more experimental features would make it less stable, you can generally get a feel for how stable a feature is by looking at it's tracking issue.

These days most tutorials/books/tools don't seem to use nightly in my experience, though many still do. But if you're primarily looking at blog posts that are pushing the boundaries of the language (e.g. by writing operating systems) I expect that most of them would be using nightly.

estebank2• 6 years ago

Stable is pinned nightly with disabled nightly features and beta backports applied. The difference might sound like semantics, but it is present and can impact you.

kbenson2• 6 years ago

I think perhaps a lot of projects start out using nightly with the assumption that within 6-12 months those features they are using will be in stable.

Whether that generally happens in reality, or people get into a loop of "well, this other new features is nice and in nightly, and we're already in it, so why not", I'm not sure.

fluffything4• 6 years ago

I think perhaps a lot of projects start out using nightly with the assumption that within 6-12 months those features they are using will be in stable.

In my experience, that is rarely the case.

People mostly use nightly features because they just need to get something done, and these features make that easier.

The tooling around Rust is quite good (rustup, cargo, docker), so if you follow the language a bit, judge which nightly features you opt-into, pin nightly version in your build environment, etc. breakage only happens when you actually have time to fix it and the fixes are simple.

So the bar for how much easier a nightly feature needs to make your problem to overcome its costs is quite low. The moment you are using one nightly feature, the cost of using more is even smaller.

monadic22• 6 years ago

I know it shouldn't be a big deal but it "feels dirty" and perhaps I should just view nightly as any other language's "stable".

There are two uses for nightly:

1. Access to unreleased features. I think I'm down to just needing this for associated_type_defaults (rfc here: https://github.com/rust-lang/rfcs/pull/2532/files). 2. Nightly allows passing extra compiler flags, which is occasionally useful for debugging things like macro expansion.

Personally I've found that nightly is pretty solid and I do tend to use it more than stable, but I certainly would not consider it dirty in itself. Furthermore if you're looking at a tutorial or a book the features it relies upon may have gotten into the stable build.

estebank2• 6 years ago

My advice is generally: nightly for development, stable for ci. If you don't rely on nightly features, nightly will still let you reap the benefits of improvements that will land in stable 12 weeks later.

sondr3• 6 years ago

I think just two years ago it was much more normal to use nightly because of some feature that you needed, this is mure rarer nowadays. I haven't used nightly in months, and then I only used it to take a look at Rocket. Unless you require something behind a feature flag (in which case you probably know what you're doing) you should be using stable.

webgoat• 6 years ago

I've been following along while building an armv7a kernel and I really like how you go back and refactor old parts to match newer rust features like your change to using cargo instead of a makefile and inlining asm files into the rust binary.

My only criticism is that sometimes there's parts in your tutorial that are missing but present in your os. When going through chapter 3/4 I think you skipped over explaining your kmalloc implementation. It's fine that you do since I feel your tutorial should be more about navigating embedded rust than learning os basics, but a note about this task being implemented at some point would've helped.

pavehawk200713• 6 years ago

Thanks for the feedback. It does get tricky to balance going too far into the weeds and keeping it above board. I think a note would be helpful as you suggested.

I do try to put lots of comments in the Rust code to describe what I didn't in the blog for those more interested in the implementation.

Animats2• 6 years ago

I'd like to see a clone of the QNX kernel, which is very tiny, implemented in Rust. It has no drivers or file systems in kernel space; those are all user processes. But it offers a POSIX-like API. Unlike L4, you don't have to run another OS on top of it to do anything.

This would be good for embedded applications on Raspberry Pi sized machines, where the Arduino environment is too weak and Linux is too much.

speed_spread• 6 years ago

Aren't there patents protecting it? I remember reading something about the fast interprocess call mechanism... Microkernels are prone to high overhead because everything requires context switching. Maybe this isn't true anymore.

spc476• 6 years ago

QNX has been around for a long time. Back in the mid-90s I did some consulting work porting some Unix utilities to QNX, and if there were patents back then, they have now expired.

Animats2• 6 years ago

US 8,434,086, "Process scheduler employing adaptive partitioning of critical process threads", is still in force until 2030. That's a more recent feature for hard real time, where you want to guarantee that some thread gets, say, 2ms of CPU time every 10ms of elapsed time. Few schedulers do that sort of thing at such fine granularity, but that's what you may need if you're controlling some actuator. Good for things like audio players, too. It's not a commonly used feature, and went into QNX some time after 2005. I remember it being listed as a future feature back then.

The fast interprocess call mechanism is what makes it all go. It's mostly about it being simple. Mach tried to get too cute, moving pages from one address space to another. Turns out that copying is usually cheaper than stalling out the CPU and flushing caches while you mess with the page tables of recently active pages. L4 just offers shared memory, dumping the problem on the next level up. QNX offers useful send/receive message functionality that goes fast. Yes, there's a penalty, maybe 20%. But you usually get that back because you're not forced to do interprocess communication with something far slower, like HTTP.

pjmlp2• 6 years ago

I guess Tock or Redox might be your options there.

bogomipz• 6 years ago

I had a question of the about a passage under the section: "Our OS's Filesystem Read Process" where the author states:

"Recall that the block driver makes a request and sends it off to the block device. The block device services the request and then sends an interrupt when it's finished. We don't really know when that interrupt will come, and we can't wait around for it. This is why I decided to make the file system reader a kernel process. We can put this kernel process in the waiting state so it won't be scheduled until the interrupt is received from the block device"

Then further down: >"Now, when we handle the block device's interrupt, we have to match the watcher and awaken it."

Is the watcher here similar to the filesystem driver at the VFS layer in Linux i.e the ext4/xfs etc? I believe this is how Linux handles it - the file system driver creates a block IO request and hands it off to the actual block device driver and when the block device driver get's invoked on an interrupt from the disk the block device driver informs the filesystem driver that the IO request is now complete correct?

RISC-V OS using Rust: Filesystems

Related Stories