Redox – A Unix-Like Operating System Written in Rust

http://www.redox-os.org/

by tilt • 10 years ago

618 202 10 years ago

redox-os.org

freekh• 10 years ago

Looks real cool. Congrats! Wonder if some of the utils like the shell would be a cool Rust project for me :)

terda12• 10 years ago

Not a fan of the name, sounds too much like Redux :/

conradk• 10 years ago

First commit for Redox was April 20th 2015. First commit for Redux (the JS library, if that's what you're talking about) was May 30th 2015.

746F74752• 10 years ago

If no one knows about your project it's name doesn't matter.

Bring to mind the old MikeRoweSoft vs. Microsoft thing.

pjc50• 10 years ago

Presumably named after the redox reaction that causes iron to rust.

iagooar2• 10 years ago

The name is quite clever, actually.

ape42• 10 years ago

Not that it matters... Apparently rust-lang is named after the rust fungus not the chemical reaction. https://en.wikipedia.org/wiki/Rust_%28programming_language%2...

systems• 10 years ago

sound a lot like redo x nobody likes to redo things .. or x'es

would you ever redo your x

khattam• 10 years ago

Yes... that's what x'es are for.

yuchi• 10 years ago

I’m not into OSes but I have to say it looks incredibly interesting and makes me want to dig the codebase. Awesome brand also!

qz_2• 10 years ago

I'm pretty impressed that 37 people are capable of writing an entire OS with a gui in under a year. Looks really cool.

teamhappy• 10 years ago

Operating systems aren't that big, and if you know your stuff, I'm sure it's not too hard to pull off. Here's an example I borrowed from StackExchange:

According to cloc run against 3.13, Linux is about 12 million lines of code. 7 million LOC in drivers/, 2 million LOC in arch/, and only 139 thousand LOC in kernel/. (http://unix.stackexchange.com/a/223753)

Edit: Would be nice to have the numbers for the latest Minix release for comparison. Does anybody know how big their core team is? (The kernel is something like 15k LOC IIRC.)

Manishearth4• 10 years ago

Btw, if anyone is interested in learning how to write an OS, and interested in Rust, check out https://intermezzos.github.io/. It's a "learning OS" in Rust, with a companion book. Incomplete, but a good start.

iagooar2• 10 years ago

Warning: I have no idea about systems programming.

I understand that the amount of lines of driver code comes from the variety of devices. But still, it looks completely unbalanced, when compared to the kernel code itself. So I have some questions here:

1) Shouldn't there be common interfaces / abstractions for most of the devices? 2) If they exist, could they be improved somehow? 3) A bit unrelated, but how fun / interesting is it to develop driver code?

pmlnr• 10 years ago

Actually, they can be really small... http://kolibrios.org/

zongitsrinzler• 10 years ago

Are you the guy that says stuff like "How hard can it be?" and "It shouldn't take that long" in meetings?

carussell• 10 years ago

Does anybody know how big their core team is?

Right now it's about 2.

jotux• 10 years ago

I worked at a company that made POS terminals on the OS development team. We had a proprietary OS based loosely on UNIX that was ~100KLOC (kernel). It was largely designed by 3-4 core developers and maintained by ~12 embedded software engineers (in addition to drivers, apps, UI, etc).

bane• 10 years ago

it's actually a pretty common task in undergrad CS programs for an individual or a small group (3-4 students) to write an *nix-like OS from the ground up, including most of the core utils, scheduler, process management, etc.

This is obviously much more full featured (includes a GUI), so this is actually a very nicely sized team for the task.

notalaser6• 10 years ago

Ours didn't have a GUI. It took us less than a year to write and the team was never larger than 6 people. It's less complicated than it seems at first sight.

There are a lot of things about machines that you need to understand, but the technology itself is relatively well-understood and sane practices are encouraged.

It blows my mind that less than twenty people can string up something non-trivial with Node.js et co.. Operating systems are trivial in comparison.

joepie91_• 10 years ago

As somebody who develops significantly complex in Node.js and considers operating systems to largely be black magic, I'd love to have a chat about this some day, and compare notes :)

nickpsecurity28• 10 years ago

Your website is interesting but a tad hard to follow due to the "organization" scheme. ;) So, what OS project did you work on in that first sentence and do you have a link to it?

jakub_h• 10 years ago

Have you heard of Oberon? That's like three people in a year. Including building the computer. ;)

ape42• 10 years ago

And rust only hit 1.0 on May 15, 2015 (I looked it up https://en.wikipedia.org/wiki/Rust_%28programming_language%2... )

jroesch• 10 years ago

Its actually much easier then it sounds, we had an undergrad write an entire windowing system on-top of implementing the rest of the OS for undergraduate operating systems at UW in a single quarter (2.5 months~).

jstimpfle• 10 years ago

Also relating to the whole thread, I think these kinds of statements are dangerous. What is a "windowing system" here? There is a huge spectrum. If a windowing system were so easy, why has Wayland taken such a long time? It's best to give a link to the project so people can see what problems were actually solved and how useful it is in practice.

I have been thinking hard about a simple plain text file format and associated tools for the last couple months. Sounds nowhere near as impressive. Yet I hope the end result could eventually make a practical difference.

nickpsecurity28• 10 years ago

Wirth and a few others went to Xerox to see their personal computer with GUI that Steve Jobs also envied. They were told they couldn't buy it. "So, I built one..." (Wirth)

http://www.cfbsoftware.com/modula2/Lilith.pdf

Lilith was a custom computer with all kinds of hardware. The mouse eventually inspired Logitech. The software included a safe language (Modula-2), its compiler, an OS (Medos-2), a relational DB, and some other stuff. The trick was safe language, modularity, and simplicity in implementation details. Two or 3 people over a few years.

Later, that turned into the Oberon series of languages, compilers, and OS's. The extensions or ports usually took 1 or 2 students months to a year or two to pull off. Last one they released IIRC was A2 Bluebottle OS. Primitive, but usable. I keep telling people to rewrite it in other safe, C alternatives to save them work. Students got it done fast so I'm sure hobbyists could too.

https://github.com/Ignat99/a2

vram22• 10 years ago

Interesting. I had read about Oberon (the OS) and probably Lilith in an old BYTE magazine issue years ago.

PaulRobinson• 10 years ago

You'd be surprised. There are lots of people writing operating systems as hobbies.

There's a wiki to get you started if you're interested: http://wiki.osdev.org/Main_Page

chillingeffect2• 10 years ago

Oh man if we could get hard realtime scheduling in early, large, critical embedded systems like medical would benefit greatly!!!$$$

jeffreyrogers• 10 years ago

Doesn't that exist with something like FreeRTOS?

chillingeffect2• 10 years ago

Yeah it looks good for realtime, but i'd like to use rust and gnu/linux/posix utilities. Do you know much abt filesystems on freertos?

TickleSteve• 10 years ago

there are many, many RTOSes out there, most very well battle-tested. There really isn't a need for any more.

0xcde4c3db• 10 years ago

While I'm not sure this fits with Rust's existing concepts of safety and correctness, it would be interesting (and not just for RT purposes) to have a language in which one could mark functions as <= some complexity (given the usual simplifying assumptions) or provably terminating, and have the compiler throw an error if it can't prove that those constraints are met. Does something like this exist?

abiox2• 10 years ago

i don't know, but wow, that's a really cool idea.

Rusky2• 10 years ago

The closest thing that comes to mind is total functional languages- the compiler doesn't deal with complexity classes, but it does prove termination. It's mostly used in dependently-typed languages and theorem provers so they can prove the type checker will terminate.

progman6• 10 years ago

Great news! Now it's worth to learn Rust just to keep up with this project.

Does Redox run in qemu only or also in VirtualBox? I tried it with all devices (USB, Network, etc.) disabled. Installation from ISO to /dev/sda worked fine but after reboot I got a hangup: "IDE Primary Master: Unknown Filesystem". Do I have to format the hardisk image with ZFS before installation?

jszymborski• 10 years ago

The book has steps, but I think it runs through qemu

stryan• 10 years ago

qemu is recommended but Virtualbox is supported. Try running grabbing the latest source and running make virtualbox.

_0w8t• 10 years ago

So I can play with it on OSX ?

Nullabillity• 10 years ago

The installer worked under VirtualBox for me.

progman6• 10 years ago

Thanks for the recommendations!

First I have to install the current Rust version. Are there any MD5/SHA256/GPG to verify the integrity? Rust's download page doesn't provide anything like that.

https://www.rust-lang.org/downloads.html

steveklabnik9• 10 years ago

http://static.rust-lang.org/dist/ has them. If you use multirust to install, and you have GPG installed, it will use them to check upon installation.

fche2• 10 years ago

How many "unsafe" declarations should one expect in regions of the code that will need to deal with untrusted data (like networks, syscalls)?

progman6• 10 years ago

I guess not very much if Redox will support open hardware.

http://www.xda-developers.com/risc-v-cores-and-why-they-matt...

http://hackaday.com/2014/08/19/open-source-gpu-released/

http://www.openfpga.org

http://hackaday.com/2014/08/19/open-source-gpu-released/

http://open-ethernet.com

Some compelling projects (GPU) failed for lack of interest. The concepts behind those projects did not fail because most of the projects were isolated. The best time for open hardware is yet to come. Redox could accelerate that.

fche2• 10 years ago

rust "unsafe" is not about proprietariness of the hardware. It's about doing lower level operations that, in order to express, the rust compiler's safety logic must be disabled.

Manishearth4• 10 years ago

I don't know about Redox, but Phil's Rust OS (http://os.phil-opp.com/) seems to manage to minimize unsafe Rust and only use it to create some low level OS abstractions that can be safely used. It's pretty neat.

bydo• 10 years ago

From the book[1]: A quick grep gives us some stats: The kernel has 16.52% unsafe code, a 50% improvement in the last three weeks. Userspace has roughly ~0.2%.

1: http://www.redox-os.org/book/book/introduction/unsafes.html

amelius2• 10 years ago

I was reading this:

http://www.redox-os.org/book/book/design/urls_schemes_resour...

and I really got interested in the "everything is a URL" idea. But then I noticed that the most important parts of this text were missing :/

Perhaps somebody can clarify here.

Manishearth4• 10 years ago

There's a bit more info here: https://github.com/redox-os/redox/wiki/URL

rch• 10 years ago

Interesting. They mention 9P, but their approach reminds me of 1060's NetKernel:

http://www.1060research.com/products/#roc

amelius2• 10 years ago

Very interesting.

The only odd part is that the modules (drivers) themselves are not referenced by URL, but only by a simple word (in the example "port_io").

Also, I wonder how we could combine drivers. For example, (theoretically) instead of using "https" as driver, we could compose it as "HTTP over TLS over TCP", and change any of those subcomponents as desired. With URLs this might become clumsy.

tlrobinson• 10 years ago

That's pretty neat. You could kind of fake it on other OSes using something like Fuse with mount points like "/http" or "/tcp".

yincrash• 10 years ago

I wonder why they decided to write their own coreutils instead of using https://github.com/uutils/coreutils

steveklabnik9• 10 years ago

That project is attempting to re-create GNU-style coreutils, they wanted a slimmer, BSD style one, from what I understand.

dschiptsov• 10 years ago

UNIX98, POSIX compliance?)

unsignedqword• 10 years ago

Apparently not http://www.redox-os.org/book/book/introduction/what_is_redox...

jpgvm• 10 years ago

The Rust implementation of ZFS interests me greatly. Going to pull this down and try hack on it.

skoink• 10 years ago

I've seen this project a few times - very impressive work! It'll be awesome to see if they can get it self-hosting (right now, I think you need to compile with a different OS).

b3h3moth• 10 years ago

"Using Rust for an Undergraduate OS Course"[0] is an interesting reading about Rust, OS and C.

[0] http://rust-class.org/0/pages/using-rust-for-an-undergraduat...

nailer• 10 years ago

Anyone have screenshots of the orbital GUI?

Manishearth4• 10 years ago

There are some redox screenshots here: http://www.redox-os.org/news/

ethanbond• 10 years ago

I'm also curious how to contribute to the GUI. I'll trawl the github in the meantime but if anyone happens to know a good point to start I'd love to hear.

Edit: Found it, in case anyone else is interested: https://github.com/redox-os/redox/blob/master/CONTRIBUTING.m...

hnmcs• 10 years ago

https://www.google.com/search?tbm=isch&q=rust+redox+gui

led me to multiple interesting websites

tweak the terms for more results, of course

iso-8859-1• 10 years ago

It is a shame that OS development is so dependant on toolchains that whatever focus an OS may have, it excludes other areas of development. For example, I very much like Genode. But to redo Genode in Rust would probably be harder than making a Unix-clone. Safe language or safe API's, it seems like you have to choose one.

cgcardona• 10 years ago

Previous Discussion: https://news.ycombinator.com/item?id=10295187

jeffdavis• 10 years ago

What is the main differentiator? Security/stability because it's written in rust?

mastax• 10 years ago

Maybe in 10 years. Right now it's more interesting/unique because it's written in Rust.

chris_wot2• 10 years ago

I feel silly I haven't o ask this, but what is the best guide to learning rust?

steveklabnik9• 10 years ago

I am biased, but the book is the most complete resource: https://doc.rust-lang.org/book/

There's also http://rustbyexample.com/

chris_wot2• 10 years ago

Hey, thanks :-)

Animats4• 10 years ago

Very nice. It was time to do this.

There are three near-term products to develop on this:

- A small router (home/small office, not data center)

- A DNS server

- A BGP server

Those are standalone boxes which do specific jobs, they're security-critical, and the existing implementations have all had security problems. We need tougher systems in those areas.

nickpsecurity28• 10 years ago

I was worried a whole UNIX might be too big a project so I gave him old school recommendations before. Essentially told him to make a split system that lets you run on tiny kernel bare metal, run UNIXy apps with different API, and let them communicate through some IPC mechanism. That way, like security & MILS kernels, people can write security critical components in isolated partitions with checks at interfaces. Already done with Ada and embedded Java. Rust is best one to try it with now.

Far as your recommendations, I like those. I'll add a web server, Redox or UNIX compatible, that's efficient enough to be deployed in all these web-enabled embedded devices. The dynamic part can just be Rust plugins or something. However, just a robust Ethernet stack, networking stack, time API, and simple filesystem could be used to implement all of yours, the web server, and more. So, I encourage people building these OS projects to stay on 80/20 rule hitting features almost all critical things use. Others can jump in at application layer from there.

Animats4• 10 years ago

A web server gets big, complicated, has lots of add-on parts, and has performance constraints. Small routers, DNS servers, and BGP servers are small, closed systems that should Just Work. You want to get them working, lock the code into read-only memory, and forget them.

burfog9• 10 years ago

In other words, like GNU HURD. That design is very very hard to make correct and fast.

Correctness suffers because the UNIX API has all sorts of interactions between different parts. This includes atomicity. It's a bear to get this right with IPC.

Performance suffers because you are unable to effectively share data structures. This too relates to the interactions between different parts of the UNIX API.

Look, there is a reason GNU HURD is slow and suffers from incompatibility. It's a cute thought experiment, dominating academia around 1990, but it's not actually fast or maintainable. Experience has proven this.

polyfill• 10 years ago

This may be helpful for the DNS server https://github.com/gtadam/rust-dns

nickpsecurity28• 10 years ago

Interesting. So will this for anyone brave enough to use SPARK and Rust:

http://ironsides.martincarlisle.com/

IRONSIDES uses SPARK to show freedom of exceptions or single-packet DOS. A translation to Rust would probably preserve most of those properties. I also thought a parallel development between Ada2012/SPARK and Rust would be interesting to see if one set of checkers catch something another misses in same piece of software.

bluejekyll2• 10 years ago

I'm working on something: https://github.com/bluejekyll/trust-dns

I'm in the midst of implementing DNSSec right now, I've got RRSIG and DNSKEY validation back to the root now. I'm working on negative query validation right now, NSEC and NSEC3. Maybe a new release in a few weeks after that is supported.

tbolt2• 10 years ago

Awesome. Really great to see this. Nice work to everyone involved

tbolt2• 10 years ago

Awesome. Nice work to everyone involved

tomlong• 10 years ago

So this is like Hurd, right?

FlyingSnake• 10 years ago

This is bloody Brilliant! The Rust community is doing a great job in delivering great projects.

Are there any concepts borrowed from past OS experiments like Plan-9, Midori, BeOS etc?

nickpsecurity28• 10 years ago

I like the home page: unusually good choice of attributes for a security or reliability-focused OS. The path usually not taken. So, for those familiar, what milestones has the project achieved since we last discussed it here?

Note: Best bang-for-buck will be getting solid networking, filesystem, time API, and crypto lib in there. People can crank out purpose-built appliances or VM's for all Internet or Web servers with... not ease but easier. 80/20 rule always best for OSS projects to get adoption & contributions up. ;)

clem16• 10 years ago

That looks impressive. It looks very clean and well put together. I will have to check this out!

kpozin• 10 years ago

I've been looking through some of the kernel code [1] out of curiosity, and I'm very surprised to see that most components have almost no internal documentation — minimal file and class comments, and even fewer inline comments. Is this typical of code for something as complex and central as an OS kernel? Are the developers planning to go back later and add documentation, or is the expectation that anyone who might need to work with this code will find its structure and details intuitive?

[1] https://github.com/redox-os/redox/tree/master/kernel/

kibwen2• 10 years ago

There appears to be a book that serves as documentation, though I don't know how complete it is: http://www.redox-os.org/book/book/

tinco2• 10 years ago

Yes, I like to read sources of big projects as a relaxing activity and almost never is documentation written in a way that a newbie can immediately catch up. The best projects will have a README.md in every directory that explains what a component or namespace is about, or nice big documentation headers. In my experience though if you really want to contribute the lack of documentation is only a small hurdle. If the code is well written then just following the code from the entrypoint on will give you a very good idea about what's going on. All the files and their names should make sense in under an hour or two.

In my opinion documentation is most important for external consumers of APIs who don't want to spend time grokking the code, and for explaining design decisions. For contributors it would be code quality.

jaltekruse• 10 years ago

I only took a brief look over the code to try to find what documentation is actually there, but I managed to find this

https://github.com/redox-os/redox/blob/master/kernel/fs/url....

On your question of how typical a lack of system-internal documentation is in systems projects, the answer from my experience is unfortunately more common than it should be. There are a bunch of usage and design patterns that cannot be expressed regardless of the language you are using. I know very little about Rust, but I work mostly in Java, which is safer than many other languages (less flexible than interpreted or dynamic languages, less dangerous than C/C++). Even java, which has interfaces and strict typing can save you some of the more mundane unit tests and input validation code, has a bunch of design patterns on top of the core language. Projects could serve themselves well to document their code well to allow new contributors to look at any of the code and reasonably modify it to fix a bug or add a feature.

ivanceras• 10 years ago

In java, the jvm is the OS. Every object in Java is allocated in the heap, and doesn't provide developers direct control of a more optimized way of allocating objects. In rust, you can specify an object to be on the stack or on the heap, rust compiler is very smart and it knows where an object goes out of scope, thus not needed anymore and insert a code on that location to drop that object and free memory. Unlike in Java, which waits for the garbage collector to drop objects.

In a nutshell, java engineering effort is directed into the jvm(jit) to provide high performance when the applications are run. Rust engineering, on the other hand is directed on the compiler figuring out where and when an object is out of scope, no matter how complex it was attached to other objects, passed around multiple functions. It will figure it out.

sambeau• 10 years ago

There is this comment :) ..

    /// Into a cow
    pub fn into_cow<'a>(self) -> CowUrl<'a> {
        /*
             ______________
            < Mooooooooooo >
             --------------
                    \   ^__^
                     \  (oo)\_______
                        (__)\       )\/\
                            ||----w |
                            ||     ||
         */
        CowUrl::Owned(self)
    }

pointfree• 10 years ago

"MIT licensed"

...and consequently limited driver support. Copyleft is permissive. There's no need for non-copyleft licensing unless you want restrictive proprietary licensing somewhere or sometime.

ci5er• 10 years ago

"Limited driver support", here, means ... you don't have access to, and the right to fork, the source code?

As an old engineer, to me, limited driver support always meant: "not that many drivers". You seem to mean: "I can't fork".

Or am I mis-reading you?

Many of those of us in the new-new world of next-gen system-integrators-acting-like-new-software-product-developer types don't always have sole control over the drivers in the/our stack necessary to deliver key features to key clients.

The "hard" open source position of the copy-left crowd incentivizes old-school pragmatic management to take a "why bother" stance and instead of open sourcing 80% and getting yelled at because it isn't 100%, just go with 0%.

Which is sad. An unnecessary. The won't deal with the very real business risk that happens when you liberally treat with zealots.

Or am I (I ask again) mis-reading you?

lolidaisuki• 10 years ago

80% isn't any better than 0%.

YngwieMalware• 10 years ago

Beautiful piece of work. Thank you to the whole development team

IshKebab• 10 years ago

Why create a new Unix-like OS? Sure, have a Unix compatibility layer, but there are so many ways to improve on Unix. Seems stupid to repeat the mistakes of the past.

Redox – A Unix-Like Operating System Written in Rust

Related Stories