Writing an operating system kernel from scratch

https://popovicu.com/posts/writing-an-operating-system-kernel-from-scratch/

by Bogdanp • 10 months ago

350 69 10 months ago

lordleft• 10 months ago

I will never not find this kind of project incredibly impressive. It’s interesting to think that Linux, after all, is really just the kernel — and yet getting that work done paved the way to getting an open source version of Unix installed on billions of machines. Great stuff!

hollerith4• 10 months ago

It is equally valid to say that Stallman's starting to write a C compiler and Unix utilities (in 1984 whereas the Linux project started in late 1991) paved the way to getting an open source version of Unix installed on billions of machines.

kimixa2• 10 months ago

I agree - there's a number of kernels that were "open source" and released at a similar time enough time to linux (e.g. 386BSD in '92) that I could see any of those winning the "community battle" and taking that space instead, but no real credible "development toolchain" equivalent until decades later.

Though I'm unsure how differing licenses might have affected this - I suspect that really early in it's development the "copyleft" nature of the GPL Linux didn't make as much of a difference, as from what I remember most commercial uses of Linux didn't come until it had already gained significant momentum.

Quekid5• 10 months ago

[Stallman/GNU] getting an open source version of Unix installed on billions of machines.

Agreed, funnily enough GNU tools/compilers also ended up getting installed on a lot of proprietary UNIXes because proprietary UNIX was mostly shit (in user space!). At least most of the ones I had the misfortune to have to work on.

diath• 10 months ago

It's even more funny and amusing when you remember that in the initial release mail for Linux, Torvals said "just a hobby, won't be big and professional like gnu".

https://groups.google.com/g/comp.os.minix/c/dlNtH7RRrGA/m/Sw...

chuckadams2• 10 months ago

He's also referred to Linux as "the GNU Emacs of all terminal emulators".

hu3• 10 months ago

Transcribed here for posterity:

Hello everybody out there using minix - I'm doing a (free) operating system (just a hobby, won't be big and professional like gnu) for 386(486) AT clones. This has been brewing since april, and is starting to get ready. I'd like any feedback on things people like/dislike in minix, as my OS resembles it somewhat (same physical layout of the file-system (due to practical reasons) among other things).
I've currently ported bash(1.08) and gcc(1.40), and things seem to work. This implies that I'll get something practical within a few months, and I'd like to know what features most people would want. Any suggestions are welcome, but I won't promise I'll implement them :-)
Linus (torv...@kruuna.helsinki.fi)
PS. Yes - it's free of any minix code, and it has a multi-threaded fs. It is NOT protable (uses 386 task switching etc), and it probably never will support anything other than AT-harddisks, as that's all I have :-(.

Someone• 10 months ago

I will never not find this kind of project incredibly impressive

I wouldn’t call it incredibly impressive. The path on how to write a minimal multi-tasking kernel has been beaten decades ago.

Writing a kernel that can boot and do a few things is ‘just’ a matter of being somewhat smart and have some perseverance. Doing it for RISC-V complicates things a bit compared to x86, but there, too, the information about initialising the hardware often is easily obtained (for example: https://wiki.osdev.org/RISC-V_Meaty_Skeleton_with_QEMU_virt_...; I wouldn’t know whether this project used that)

I think the author agrees, given (FTA) that they wrote “This is a redo of an exercise I did for my undergraduate course in operating systems”

It’s work, may be nice work, but I think everybody with a degree in software engineering could do this. Yes, the OS likely will have bugs, certainly will have rough edges, but getting something that can multi-process, with processes shielded from each other by a MMU isn’t hard anymore.

gertop• 10 months ago

I think everybody with a degree in software engineering could do this

Ideally this would be true, but it hasn't been my experience at all. At least with American graduates, I can't speak to other countries.

d_sem• 10 months ago

Is there a book one can read to learn how to create one?

flamedoge• 10 months ago

To you maybe. The subset of population that is even interested, smart, persevering to do this is extremely tiny.

ludicity2• 10 months ago

I am 99.99% sure that less than 20% of Australian graduates could do this, and honestly I wouldn't be surprised to hear that the actual answer is <1%.

I was studying at Monash, which is considered a solid university here, and holy moly are the standards low. I had classmates in the second year of my machine learning postgrad asking me things like "What is machine learning?", and they all graduated and found jobs anyway.

WD-423• 10 months ago

Zig really is amenable to OS development. And so is RISC-V. I started this same exercise except with x86 first. I quickly got annoyed with all the legacy boilerplate required. RISC-v doesn’t have any of that. It’s so much faster to get up and running. Here’s my bad zig: https://github.com/Fingel/aeros-v

toast0• 10 months ago

with x86 first. I quickly got annoyed with all the legacy boilerplate required.

IMHO, if you use a reasonable bootloader, you don't have too much boilerplate. Multiboot loaders do leave you in real mode, and most people want to be in protected mode, so you have to set up some tables and do a jump, but that's not that much boilerplate. There's a bit more stuff if you disable legacy interrupt controllers (which you probably want to) but it seems to me being able to maybe run on a regular pc is worth it (caveats about console interfaces apply... my hobby OS needs bios boot and uses some vga things that I found aren't well supported... serial console is easier, but lots of computers don't have a serial port either)

ethin3• 10 months ago

You still have to set up the GDT, TSS, all that other legacy stuff that isn't actually used but it's still there. And if you set it up incorrectly, your punished for it with triple faults and the like, even if nobody even uses anything like the GDT or TSS these days.

josephg• 10 months ago

I’d like to have a go at this. What are you using to run your risc-v kernel? Just Qemu or can you recommend any real hardware?

WD-423• 10 months ago

Just Qemu. I used this as a guide: https://operating-system-in-1000-lines.vercel.app/en/ It's C so translating to Zig is pretty fun.

pjmlp• 10 months ago

Naturally, it is basically revisiting the safety of Object Pascal and Modula-2, and repacking it into a C syntax.

C was never special, beyond being widely available thanks to UNIX original licensing.

chris_wot2• 10 months ago

How does one get their hands on low-cost RISC hardware?

Rzor• 10 months ago

There's this $10 board on Aliexpress called Milk-V Duo S. It's been popping up on my recs every now and then. Looks interesting.

https://aliexpress.com/w/wholesale-Milk%2525252dV-Duo-S.html

Taken directly from its description:

Milk-V Duo S is an upgraded model of Duo with an upgraded SG2000 master with a larger 512MB RAM and extended IO capability
Onboard WI-FI6/BT5(Model Milk-V-Duo-S-512M-Basic/Milk-V-Duo-S-512M-eMMC does not have this function)
USB 2.0 HOST port
100 Mbps Ethernet Portwith PoE Support (via PoE HAT)
Dual MIPI CSI with ISP
The device also supports switch between RISC-V and ARM boot via a switch

hackerInnen• 10 months ago

I think there are dozens of boards by now, but what I found interesting and pledged to is this:

https://www.kickstarter.com/projects/starfive/visionfive-2-l...

I don't have a first version visionfive2 myself, but i heard good things about it and the eco-system sounds like it's growing. There are still rough edges. Lets hope they ship soon (tm)

Edit: What I currently have for my personal projects is a PolarFire SoC Discovery Kit. That's a quad-core RISC-V system with FPGA embedded. Maybe too expensive and not for everyone (130 bucks, fun fact: The dev board is cheaper than the chip itself).

https://www.microchip.com/en-us/development-tool/MPFS-DISCO-...

Also the microchip documentation, toolchain and so forth feel really crappy and old, but once you get used to it, it's not that bad and actually getting to run bare-metal risc-v code is easy. There are easy examples for linux and bare-metal

yencabulator2• 10 months ago

Maybe just run an emulator on your x86/apple, for now? It'll probably be faster than actual hardware, and it will for sure be faster to develop a kernel with.

https://www.qemu.org/docs/master/system/target-riscv.html

balder19912• 10 months ago

This has risc-v: https://www.raspberrypi.com/products/raspberry-pi-pico-2/

chris_wot2• 10 months ago

Thank you all for your excellent responses. No thanks to the fools who downvoted this.

ktosobcy• 10 months ago

El acceso a la presente dirección IP ha sido bloqueado en cumplimiento de lo dispuesto en la Sentencia de 18 de diciembre de 2024, dictada por el Juzgado de lo Mercantil nº 6 de Barcelona en el marco del procedimiento ordinario (Materia mercantil art. 249.1.4)-1005/2024-H instado por la Liga Nacional de Fútbol Profesional y por Telefónica Audiovisual Digital, S.L.U. https://www.laliga.com/noticias/nota-informativa-en-relacion...

Of FFS...

one thing is using http but another this one…

flykespice2• 10 months ago

...what? a football organization in Spain has the power to restrict internet access to the entire country?

WJW• 10 months ago

Football is really big in Spain dontcha know?

carlesfe• 10 months ago

Yes, that is exactly what is happening

lioeters• 10 months ago

I heard about this situation in Spain where Cloudflare (and maybe others) are blocked by mistake due to being associated with some sites streaming football.

How are people getting around this problem, I guess VPN? I mean, some of the wrongly blocked IPs are needed for work I imagine.

fransje263• 10 months ago

initiated by the National Professional Football League and Telefónica Audiovisual Digital, S.L.U.

These people are criminals..

NaOH• 10 months ago

Submitted separately by the author:

https://news.ycombinator.com/item?id=45236479

Copying their comment to here:

I have redone the classical exercise of writing a tiny OS kernel with time sharing, which manages a couple of user threads. My goal was to experiment specifically on RISC-V + OpenSBI. Additionally, I wanted to explore Zig a little bit, so that was the language used instead of the traditional C, but it should be straightforward how to do the same experiment in either C or Rust.

It's definitely very rough around the edges, and it's more of an experiment and an intro for people who want to go through step 0 of learning OS kernel development and computer architecture. Nevertheless, I hope it is still a fun experimental thing to play with over the weekend!

The full walkthrough and the GitHub link are available at the link posted!

kristoff_it2• 10 months ago

For those who want a "packaged" experience doing this same kind of work, there's "Operating System in 1000 Lines of Code" (https://operating-system-in-1000-lines.vercel.app/en/), I followed it a while ago in Zig (which meant translating C code snippets to Zig) and had a good time. I have the code and VODs here: https://github.com/kristoff-it/kristos/

fransje263• 10 months ago

What would be the level of Zig knowledge needed to do this exercise?

Or, asked differently, with no knowledge of Zig, what would be a realistic approach to trying this out? (Assuming an interest in learning Zig, with a background in C++)

ozgrakkurt• 10 months ago

I could easily understand stdlib code of zig after building some stuff for 2 days. Would be different for everyone but it is fairly easy

kristoff_it2• 10 months ago

It's definitely possible to hit the ground running you have some knowledge of systems programming. I would say that the Zig version is much easier to understand than the C version provided by the book because the Zig one uses `packed structs` instead of bit operations in some places, everything else is roughly the same.

Isamu• 10 months ago

Always a fun and interesting thing to try.

Also go ahead and try hard things like your own cryptography. The advice “don’t roll your own crypto” just means don’t DEPLOY something that is not battle tested, because it is very easy to make BIG mistakes in that area. Always feel free to experiment and explore.

We need more operating systems, more choice.

jackdoe2• 10 months ago

nice!

i love how accessible riscv isa is, the docs are amazing, there are tons of examples, and tons of emulators, the uncompressed machine code is quite readable.

i am writing a book for my daughter and am also working on a small os with forth and timesharing https://punkx.org/projekt0/book/part1/os.html and i dont think i could've done it on x86, learning both forth and riscv assembly on the way, its super fun.

honestly if you ever wanted to make a toy os from scratch there is no better time to start than now (besides maybe 1980), get a cheap riscv like rp2350 and just upload the relevant sections of the docs to claude and it will help you if you get stuck.

balder19912• 10 months ago

I just checked the prices for my country and I’m surprised how cheap this is. I’m 90% sure I’m buying one just to play around with.

jackdoe2• 10 months ago

and the datasheet is amazing https://datasheets.raspberrypi.com/rp2350/rp2350-datasheet.p...

there is an issue that language models have seen enormous amounts of rp2040 code and docs, and it they get properly confused a lot, so you should just pdf print relevant pages and use them in-context.

Writing an operating system kernel from scratch

Related Stories