I now have Rust code for executing executables

http://jvns.ca/blog/2013/12/19/day-45-reading-elf-headers/

by jvns • 13 years ago

145 22 13 years ago

thinkpad20• 13 years ago

Very cool stuff. But do I understand correctly, that Rust doesn't have any built-in functionality for starting executables? How did a language which the start was meant for low-level systems programming, go so long without having that built in? I guess one could call into C libraries to do that, but that seems quite unpleasant...

makomk• 13 years ago

What you're missing is that she's writing her own OS in Rust, which means that the OS functionality that Rust would normally use for that doesn't exist unless she writes it.

Ygg2• 13 years ago

As makomk noted, you can't call OS from the OS itself. She probably had to throw away most of libstd like zero.rs does for Rust to work at this low level.

jvns3• 13 years ago

Yup, all the standard library. If you look at the top of main.rs [1], you'll see

#[no_std]

which means "don't use the standard library"

Instead of the standard library, I use rust-core [2], which provides some of the standard library's features (like the ability to allocate memory).

[1] https://github.com/jvns/puddle/blob/master/src/main.rs

[2] https://github.com/thestinger/rust-core

sitharus• 13 years ago

This whole series has been fascinating. I've never had the inclination to write my own OS, but I love seeing what it entails.

frik• 13 years ago

Yes, really interesting blog articles. An OS written in Rust sounds like a good idea. I am seeing forward to her next articles :)

Btw. the OSDev.org Wiki might be helpful: http://wiki.osdev.org/Main_Page

jerf• 13 years ago

The enthusiasm is infectious. I find it helpful to step back every so often and remember just how staggeringly much it is we take for granted.

dnautics2• 13 years ago

I could also make up my own convention for system calls, but that seems unnecessary.

Is there possibly a utility in at least using your own interrupt for system calls, so that you could implement a linux compatibility layer down the line? Linux syscalls are a little bit odd relative to what you would generally do in the modern era of tons of memory.

jvns3• 13 years ago

Maybe! Why are Linux syscalls odd? How would you do it instead?

Locke16892• 13 years ago

Given that you're probably writing an x86 kernel, I'd suggest looking into the SYSCALL/SYSENTER instructions.

dnautics2• 13 years ago

I could be wrong about this but if my memory serves me correctly, linux syscalls put the first argument in AX, second argument in BX, third argument in CX, and fourth argument in DX. BSD syscalls do the saner thing and just put everything on the stack.

KMag• 13 years ago

There's probably more utility in implementing ptrace. One could then write a program that spawns off a ptrace'd child process and intercepts all syscalls and translates them from Linux syscalls to your OS's syscalls.

As a bonus, ptrace would allow you to host a slightly modified copy of your kernel in userspace, similar to User-Mode Linux. (Or you could modify a copy of User-Mode Linux to run on top of your kernel, as a heavier weight version of the syscall translation layer mentioned above.)

Other advantages of implementing ptrace include some debugging and sandboxing tools. Note, however, that if sandboxing a multithreaded process, if the sandbox intercepts a syscall where some of the arguments aren't in CPU regisetrs, the sandbox needs to pause all of the other threads in the sandboxed process, otherwise they may modify "safe" syscall arguments to be "unsafe" syscall arguments between when the sandbox inspects them and when the kernel actually processes them.

skywhopper• 13 years ago

Wow. The openness about the learning process and the daily progress updates make for thrilling reading. And I'm learning something at the same time! I genuinely admire the tone of the posts and wish I took the same attitude to learning new things. Going from "I don't know how to do something" one day to "here's how I figured that out" a few days later is infectiously fun to read. Can't wait to read more!

groby_b• 13 years ago

This is so much fun to read. I've written/contributed to a few OS's, and it's so awesome to re-experience the joy of doing this for the first time again.

You know you're hooked once you start thinking "I wonder if she accepts pull requests" ;)

jvns3• 13 years ago

I do!

The main issue there is that my build process is a bit broken -- my version of clang is not quite right somehow, and so I can't build properly without making some janky changes to rust-core.

gwu783• 13 years ago

Like the others, I'm really enjoying these posts. She seems to have a healthy curiousity for how things work, and a nice low level focus. This seems so rare these days.

But I have a suggestion.

Instead of writing your own kernel, what about stripping down someone else's? I imagine their would be great value, not to mention great fun, in reducing the size of, say, the Linux kernel, taking out things that are not absolutely necessary.

As Professor McIlroy, the man behind pipes and tr, I believe once said, "The hero is the negative coder." What that means is it's not the coder who adds features that is a hero. It is the one who fixes bugs and removes unneeded code.

I know someone, who is probably one of the best C coders alive today, who has asked for a Linux kernel under 200K. Wanna be a hero? Reduce the size of a large executable, e.g., a kernel. It's a challenge and a worthy goal.

lmkg• 13 years ago

She wants to learn kernels, and she wants to learn Rust. There are no extant kernels written in Rust, so she's making one.

Contributing to an existing program (positively or negatively) may be more valuable, but I would say this project is more ambitious. Also, if her goal is knowledge of the system, it's more instructive to build from the ground up rather than start with a system where you can always take parts of it for granted.

gwu783• 13 years ago

"... where you can always take parts of it for granted."

A truly curious person tends not to do this. The point of using a system is that you can take it apart to see how it works.

bjz_• 13 years ago

What she is doing is similar to implementing her own Lisp or Forth interpreter. It does not matter whether it will ever be used in production, what matters is learning whilst building, as opposed to by wrote.

You do mention that working on somebody else's construction would be useful, and would give a different perspective on the problem, but that process emphasizes different skills. Treat her wheel reinvention project as an investment in her future as a programmer and a systems designer. Re-building things that others have made to discover how they work is one of the first, vital steps in the process of developing new ideas and approaches to problems.

I would also add Rust offers some really great semantics, and it's well worth the trouble to see how a kernel would look in it. It's been a while since we've had a new, reasonably well known language that is truly capable of this level of systems programming.

gwu783• 13 years ago

"It's been a while since we've had a new, reasonably well known language that is truly capable of this level of systems programming."

You mean, like C?

Locke16892• 13 years ago

Well, your friend should keep a closer eye to the research literature ;) A number of projects have been launched to create smaller, leaner OS kernels.

This is particularly important in HPC applications as the kernel is a primary source of latency in computation. One of the major works in this area is (shameless plug) Kitten OS[1], which is a stripped down Linux kernel meant mainly for HPC.

However, I don't think recreating or hacking on Kitten is a good task for someone who wishes to learn kernel programming, Rust, or both. The best way to learn how something works is to remake it yourself.

[1] https://software.sandia.gov/trac/kitten

I now have Rust code for executing executables

Related Stories