After 5 days, my OS doesn't crash when I press a key

http://jvns.ca/blog/2013/12/04/day-37-how-a-keyboard-works/

by jvns • 13 years ago

259 88 13 years ago

voltagex_• 13 years ago

I really like the step by step list - nothing's been removed and I can see exactly how frustrating/enlightening the process was!

lonewolf3• 13 years ago

Agreed- the frustration was more than enough!

shubhamjain• 13 years ago

Would echoing the chracter on screen require a seperate video driver to mark the pixels that make up the character ? Considering how hard is to make the keyboard driver, wouldn't the video driver be many folds harder?

Sanddancer• 13 years ago

Most video cards (some newer ones have stopped shipping with it) have, baked in, a text mode buffer. It's included in the Vesa Bios Extensions.

ghewgill• 13 years ago

PC video systems support text mode, where making a character appear amounts to simply writing the correct bytes to the correct memory mapped video buffer address. Graphics mode is similar but with more bytes (say, four per pixel).

stephen_g• 13 years ago

Text mode is super easy - you just set bytes in the correct memory location (0xb8000000 if I remember correctly).

Writing an actual graphics driver is tricky - although there are some ways to get decent frame buffer modes (like VESA Bios Extentions), to do things really well requires thousands of lines of code before you can even get a pixel on the screen, and is different for every card.

It may be a bit easier now - you could look at the kernel mode setting code in the Linux kernel that exists now for Intel, nVidia (through Norveau) and AMD cards that didn't exist a few years ago when I was dabbling around in OS dev. Still, it would be a massively difficult task. You'd probably want to use the full Mesa stack and open source drivers that use it if possible, but writing the runtime to support it would be a big task.

sown2• 13 years ago

Writing an actual graphics driver is tricky

Does that include things that are not in text mode, if I recall the name correctly? Sometimes I just want my tiniest netbook to display the normal 80x40 resolution but it keeps putting the characters in what I think is video mode

gizmo686• 13 years ago

If I remember my go at writing an `OS` you can just ask the BIOS to print the string for you, until you switch to protected mode. Even then, you could jump between protected and real mode to use the BIOS and still get the full address space.

EDIT: Nevermind, that still required putting the string in a magic location.

pbsd• 13 years ago

0xB8000, to be precise. This is 16-bit 8086 legacy, where addresses were 20-bit and segmented.

The most fun mode was mode 0x13 from INT 0x10, with its fantastic 320x200 resolution and 256 colors. That one had the buffer at 0xA0000.

exDM694• 13 years ago

Text mode is super easy - you just set bytes in the correct memory location (0xb8000000 if I remember correctly).

In addition you have to be using I/O ports if you want to move the blinking cursor and other related stuff.

Writing an actual graphics driver is tricky - although there are some ways to get decent frame buffer modes (like VESA Bios Extentions), to do things really well requires thousands of lines of code before you can even get a pixel on the screen, and is different for every card.

This is particularly painful because you have to switch back to real mode to be able to use VESA and other Bios interrupts.

Thankfully it's not required if you're using a good bootloader like GRUB (which you should if you're doing an OS project, writing another bootloader is just more arcane hardware to work around). The multiboot specification allows you to ask for a video framebuffer from the bootloader, and you'll get a physical memory address where the linear video framebuffer resides. I bet UEFI has something similar too.

So these days, with the help of your bootloader, doing graphics on a bare metal kernel is as easy as using the text mode.

vertis• 13 years ago

Nothing quite like trying to write a complex system from scratch to realise just how many giant shoulders you stand on every day.

Trufa2• 13 years ago

I love this article: https://plus.google.com/112218872649456413744/posts/dfydM2Cn...

ColinWright2• 13 years ago

Now (re-)submitted as an item in its own right:

https://news.ycombinator.com/item?id=6853813

vacri• 13 years ago

To be honest, my first thought on seeing the headline was a particularly bad linux horror story...

ChuckMcM2• 13 years ago

I enjoyed this. It brought back memories of bringing up BSD2.9 on the PDP 11/55t we had in the lab. That particular PDP 11 was pretty rare (it had some memory split features to improve performance for computational tasks) and poking around in the kernel to get to the point where init was started. Nothing quite like that thrill, a mixture of power, and 'oh crap this is going to be a lot of work'.

Locke16893• 13 years ago

What she's doing is surprisingly a lot harder. Getting a modern IA-32(e) architecture into the 64 bit mode we all know and love is INSANE.

ChuckMcM2• 13 years ago

I'm not going to argue on 'harder' or 'not as hard' as such things are often personal measures that are difficult to quantify. I will however point out the meta fallacy of even thinking about these things in terms of 'not hard.'

The truth is, computers have become exceptionally complex. That is one of the reasons I've been building a medium complexity standalone system (ARM Cortex M based) to give folks something that is somewhere between 8 bit Arduino type experiences and 64 bit IA-64 or even ARM Cortex A9 level complexity. I realized when I started finding ways to teach my kids about computation that I was very lucky to have things like PDP-11's, VAXen, 68000's, and DEC-10s to play with which did not present this huge wall of complexity that needed to be scaled to get to the fundamentals. My target is a self hosted 'DOS' style Monitor/OS for the Cortex M4 series. Complex enough to host its own development environment and tools, but simple enough that you can keep it all in your head at the same time.

forktheif• 13 years ago

I'm guessing that a lot of the difficulty comes from the fact he's creating an OS for a PC, which is a pretty quirky and inconsistent architecture.

wikwocket• 13 years ago

The blog author is not a 'he' but a 'she.' Her name is mentioned through the article.

And I'm guessing that the difficulty comes from the fact that she's building a freaking operating system from scratch... ;)

revasm2• 13 years ago

Is the gender of said author supposed to be curious or worthy of attention? I'm not intending to be impolite, but this is the third time in several days that HN posters have brusquely corrected a gender pronoun in relation to a submission of nondescript importance.

mwill• 13 years ago

Reading this gives me a yearning in my belly to drop everything and hack away at a toy OS for a while

Jemaclus• 13 years ago

I did this last week in the three days before Thanksgiving. I got stuck at the bootloader. Oops? :)

asperous• 13 years ago

She better not keep this up, first it's just tinkering around and then boom... it's a slippery slope to creating a whole kernel.

jlawer• 13 years ago

This brings back memories...

I remember fresh out of high school following a few tutorials and writing a basic kernel & code for my old Pentium MMX and reading through the intel developer manuals for the 386 processor , and all the complexity involved that was inherited from the 8086 & 286 processors. Was a fantastic learning exercise... though I would suggest against using Pascal as the implementation language.

PhasmaFelis2• 13 years ago

I've never done OS programming, but something about this just tickles me. I think it's the way she cheerfully acknowledges screwing up constantly but still gamely keeps tackling the problem. To admit your mistakes without shame, and try again without bitterness--to lose your ego, in other words--is something I certainly aspire to, as a programmer and a human being.

joshguthrie• 13 years ago

In a context where everything "low-level" is handed to us without a "how-to", looking under the hood to build your own thing is always a pleasure, be it a new programming language or a whole OS.

To attain this state of zen-itude, just look up something that you'd like to do where documentation is hard to come by ("how to write your own shell from scratch" is a funny one if you can want to learn a LOT about UNIX) and start your own project.

Related: とあるOS is a nice hobby OS too: https://github.com/klange/toaruos

exDM694• 13 years ago

Another good read and really gives the insight on how much time and effort it takes to make anything happen on bare metal.

@jvns: SPOILERS AHEAD, DO NOT READ FURTHER IF YOU WANT TO GET TO THE HEART OF THIS PROBLEM YOURSELF

As for your problem with the keyboard interrupt firing only once, I think (after a quick glance over the code) that you do not signal "end of interrupt" (EOI) message to the 8259 PIC chip after you have received an interrupt. I faintly recall having this same issue in my ordeals with the PIC chip, if you don't signal EOI, the PIC won't fire the next keyboard interrupt.

Although I think I was working with 8253/8254 PIT (Programmable Interval Timer) chip at the time but I think the same applies to the keyboard interrupt too, it's being handled by the PIC too.

Here's the relevant part from my project: https://github.com/rikusalminen/danjeros/blob/master/src/arc... https://github.com/rikusalminen/danjeros/blob/master/src/arc...

I'm sorry to spoil your "fun" debugging it.

Dealing with any of this is a fun exercise in being close to the metal but unfortunately has no application in modern computers because these days the built-in IO-APIC/LAPIC has replaced the legacy PIC chip.

kanamekun• 13 years ago

Spoilers for a bug report... that's a new one! Very thoughtful. :)

tehwalrus2• 13 years ago

I’m seriously amazed that operating systems exist and are available for free.

Made me smile :)

unfamiliar• 13 years ago

Press keys. Nothing happens. Hours pass. Realize interrupts are turned off and I need to turn them on.
THE OS IS STILL CRASHING WHEN I PRESS A KEY. This continues for 2 days. Remember that now that I have remapped interrupt 1 to interrupt 33 and I need to update my IDT.

I have issues like this a lot (i.e. forgetting I've tweaked something and wondering what is causing weird effects). Any advice on how to avoid it?

zenojevski• 13 years ago

Use a rubberducking text file.

Every time you think, write down a line or two delineating your current reasoning and your debugging info. When you make changes, highlight them to mark them as pending ("need to update IDT"). You can also categorize, tag lines/pieces, and so on.

When you're stuck just backtrack this file, whether it is a hour or two days from now, and you'll avoid all of this.

exDM694• 13 years ago

I have issues like this a lot (i.e. forgetting I've tweaked something and wondering what is causing weird effects). Any advice on how to avoid it?

Use your version control, commit more often and use more descriptive commit messages, along with branching/merging strategies to avoid putting not-production-ready code you are unsure of to master.

maaaats• 13 years ago

Commit before changing stuff. Easy to roll back, and to see what changes you have done lately.

bedspax• 13 years ago

and with two keys pressed at the same time?

Toenex• 13 years ago

Working at the layer below is a very useful exercise. As part of my undergrad course in EE in the late 1980's we had to design and simulate a 4 bit micro-processor using logic components (essentially NAND gates). I got completely immersed in that project. Not only did it force me to understand a complete, albeit simple processor but actually building it gave me a lot more confidence in writing code. Things have moved on a lot since then and there are many layers of abstraction meaning that most developers don't need to understand the hardware, but having a decent 'programmers model' for the OS can only be a good thing.

christianmann• 13 years ago

Was it a full microprocessor or just an ALU? In my Digital Design class we created a 4-bit ALU with about 8-9 opcodes and one register.

chinpokomon2• 13 years ago

I built a 4 bit microcontroller the same way around 2000. We flashed the gates to an Altera EPROM and other than external memory chips, the state machine, ALU, and other sub systems were assembled using logic gates we burned. We supported 16 instructions and could access 16 words of memory. That was enough to code a "password" based security system that would either grant access by raising an external line or it would output an alarm state if the code was invalid. As I recall it could only handle about 25Hz max. Still one of the most influential projects I worked on in college.

topbanana• 13 years ago

How do OS developers typically debug this sort of thing? Do they attach external hardware to step through, or is it serial logging?

dmytrish2• 13 years ago

From my small OSdev experience: write printf before anything else (it's not that hard, just write bytes into *0xB8000). This can be done even before getting any interrupts working, fortunately (without interrupts any input is almost impossible, I don't want to talk about polling).

pwg3• 13 years ago

I found a race condition in the Linux kernel once (0.8x days) that was partially triggered due to an off brand 80387 (FPU) chip using the ibm-pc speaker hardware.

The cycle was: 1) insert ioout to turn on speaker at code line X; 2) compile kernel 3) cause lockup 4) is speaker on or off? 4a) on, move ioout to a later code section; 4b) off, move ioout to an earlier code section;

Essentially, a binary type search through the code, using the on/off state of the speaker hardware to narrow the search. Turned out a lock needed to be one line later in ll_read_write_page.c (if I remember that file name correctly).

So, the answer to "how do you debug" is: "with anything you can make use of".

mediumdave2• 13 years ago

Back in the day when I was playing around with osdev, I ran my code in the Bochs emulator. WAY easier than running on actual hardware, especially starting the emulator was 10x faster then booting on an actual PC.

I do remember the excitement of getting interrupts to work - good times. Getting task switching to work was magic.

cgh• 13 years ago

She mentions in the article that she's using Qemu.

mediumdave2• 13 years ago

Ah - I missed that.

True emulators like Bochs are still potentially a win if they provide fine-grained debugging features. Bochs was nice because you could single-step through assembly and view contents of registers/memory quite easily. You could also attach gdb to it (although this was somewhat flaky.)

raverbashing2• 13 years ago

For some reason HN ate my previous comment

Doing OS things is hard. In x86 it's extra hard

x86 is one kludge on top of another. Like interrupt chaining

These issues remind me of making a Sound Blaster card work on DOS (I was making a program to play a simple file in Pascal/C)

It's very complicated, and several trials to make it work (and no Stackoverflow, Wikis, etc)

dmytrish2• 13 years ago

That's very true from both compiler and OS point of view. Intel machine code is so kludgy: variable-length codes, subtle differences in prefixes in different modes, exceptions in register handling all over the place, plenty of addressing modes for memory access, special machine codes for a qualified register (`mov %ax` code differs from `mov %cx/%dx/%bx/...`), more lengthy encoding for more often uses (like `mov (%esp)` requires SIB byte whereas it's not true for all other registers) and so on.

From OS perspective, it's also overcomplicated: switching from real to protected mode is an exercise in doing every action in a long chain properly and in the right order or the whole undertaking fails. Remapping IRQs into interrupts that do not mess with the CPU internal exceptions, weird legacy memory layout with BIOS, a slew of CPU modes (real/protected/PAE/long/etc), crazy format of entries in Interrupt Descriptor Table (where four bits of a pointer may be stored separately from the main part), where bits are often used differently for different contexts; redundant built-in features like poor multitasking support (Linux and other sane kernels try to avoid it as much as possible, but it's still is not possible to go into kernel mode from userspace without filling kernel stack pointer into a TSS entry), memory segments (in protected mode, where a pointer can address the whole virtual memory unlike real mode), four privilege rings (whereas all modern OSes use two: 0 for kernel space and 3 for userspace).

All that does not feel like it should be and often it only gets worse.

hansjorg• 13 years ago

Going from Motorola 68k to x86 with few preconceptions in the nineties was quite a shock after slowly having been convinced that the grass actually was greener.

On the x86 side I was surprised to find an unnecessarily cryptic assembly language, little-endian words and a byzantine memory model for starters.

Hacks upon layers of hacks and not the intuitive and aesthetically pleasing solutions seems to win out in most cases.

pwg3• 13 years ago

switching from real to protected mode is an exercise in doing every action in a long chain properly and in the right order or the whole undertaking fails

Very true, and while the X86 is patch upon patch upon patch, in the designers defense, the intent on any 286+ Intel CPU was simply to was to use "real mode" to provide a bootstrap CPU to use to setup enough of the protected mode data tables to be able to actually enter protected mode. The intent was that entering protected mode was a one way, boot time only, street. Could they have made it simpler, maybe, but they were also viewing it as a one time effort that once you got the switch implemented, you didn't need to touch that code again.

[edit for clarification]

Symmetry• 13 years ago

Yeah, I'll think pack on this article whenever my PIC32 fails to do what I tell it and think "It could be much worse."

Locke16893• 13 years ago

Doing OS things is hard. In x86 it's extra hard

Agreed, 1000 times over. To the author: WELL DONE.

The jumps between real mode, v8086/protected mode, and long mode are absolutely mind boggling.

Here's the best part that I remember from implementing Intel VMX support in a VMM: once you've booted the VMM and are ready to do a VMXON to load a guest, you will realize you have a problem. Specifically, Intel realized a while ago that "real mode" sucks and is obviously not the "real mode" that actual applications expect to run in, so they introduced v8086 mode in addition to real mode.

The problem, of course, is that booting an OS still requires real mode, but you can't run a guest in VMX in real mode, only v8086 mode. But all those things that Intel "fixed" in v8086 mode? Yeah they will totally keep you from booting.

So you're chugging along in v8086 and your guest does something that it should absolutely be able to do and BOOM -- you've faulted. So what's the solution? Write your own emulator for a Real Real Mode(tm) in your host, capture the faults, and re-enter after rewriting the guest's state.

Abso-frickin-lutely insane.

pwg3• 13 years ago

Specifically, Intel realized a while ago that "real mode" sucks and is obviously not the "real mode" that actual applications expect to run in, so they introduced v8086 mode in addition to real mode.

Actually, no.

Real mode is your shiny new i7 quad+ core CPU pretending to be nothing more than an 8086 from 1978. It does that because upon initial power up, none of the data tables required for protected mode operation are setup. So there's a chicken and egg problem at power-up. You need the protected mode data structures setup, but they are not present yet, so you have to start somewhere. Real mode on anything 80286+ is intended as a way to provide a CPU that can run a program that can setup the protected mode data structures so that protected mode can actually be entered.

v8086 mode was introduced in the 80386 chip as a way to provide hardware support for visualizing multiple real mode processes/applications while remaining in 80386 protected mode. I.e., it was some amount of hardware support for what VMWare or VirtualBox provides today. The faulting into protected mode on illegal access is/was intentional. To remain "protected" the code running in v8086 mode can't be allowed to do things that might bring the system down. So v8086 faults those accesses over to protected mode, and the assumption is that there is a protected mode handler that performs the operation (or denies it) on behalf, safely, so the system can continue to chug along. The capture/rewrite state is the intended design consequence.

An extremely good book on the 80386/v8086 design and how to make use of it is: "Advanced 80386 Programming Techniques" by James L Turley, Osborne Publishing, 1988. A very easy read, and extremely well written (assuming you can find a copy).

optymizer2• 13 years ago

  >  This continues for 2 days

This thing made me scoff at the article the most. I've written a unix-like OS for ARM 2 years ago as a final project for an OS class I took on campus. There were 20 other people in the class, and all of us forgot to update the IDT at some point. It was a common mistake, so for this article to emphasize it like it's some bug that takes 2 days is just lame.

Imagine if my blog post would talk about my experiences with building an HTML page, and how it took me 2 days to change a <p> element to a <div>. Let's get serious here.

How long will it take the author to write a circular buffer to read and write characters to the UART? A month? That says nothing about the difficulty of the actual task.

You know, when it comes to writing operating systems, I like to think of this quote: "it's hard, it's harder than it looks, but it ain't THAT hard" (I first heard it when the commentators were laughing at Chris Andersen was struggling to dunk in an NBA Dunk Contest).

If you're interested in writing an OS, make sure your primary source isn't some blog post. My textbook was Tannenbaum's Operating System book, and it's a very solid book that I can recommend. Tannenbaum knows operating systems. He's not going to tell you how hard it is. He's going explain all the details to you and, when you understand them, you'll think how easy kernels really are.

Good luck!

vidarh• 13 years ago

like it's some bug that takes 2 days is just lame.

Unless she was spending full time on it, it was not some bug that takes 2 days.

At the same time: If you've never come across some trivial, stupid little bug that has left you stumped for far longer than it should have, you're either superhuman (or have selective memory), or you're a total beginner.

Your dismissive tone is really offputting.

abshack• 13 years ago

When I did Operating Systems in University (in a group), I remember distinctly only 3 bugs that we had difficulty debugging:

* re-using a loop variable from the outer loop in an inner loop (damn you `int i`!)

* putting a char[256] on the stack when the stack size was only 256 bytes and getting stack corruption (this one wasn't me, thankfully)

* signed vs unsigned code being passed between user mode and kernel mode causing problems (no idea why it was causing issues; i just made it all unsigned and the issue went away)

In hindsight, -Wall would have probably caught everything and saved a couple hours of "wtf" at 2-3 in the morning.

Often the simplest things are the hardest to discover.

optymizer2• 13 years ago

  > THE OS IS STILL CRASHING WHEN I PRESS A KEY. This continues for 2 days.

In case you didn't read the article.

PhasmaFelis2• 13 years ago

Earlier in this thread, I said that shamelessly admitting your mistakes is a virtue. People like you are the reason it's such a difficult virtue to cultivate.

The lesson conveyed in your post is "if something is hard for you, keep your mouth shut about it, or someone will laugh at you." That is a fucking terrible lesson and it needs to be stamped out and destroyed. I'm still trying to unlearn it.

optymizer1• 13 years ago

I'm not saying that having a bug is something to be ashamed of. I am pointing out that the page is making a big deal out of a small issue, in order to argue a bigger point: operating system development is hard. The whole page screams with: "look how hard this is!".

The lesson conveyed in my post should be that this author is making it look harder than it actually is, and if you want to write an OS, you shouldn't feel like it's an unsurmountable task. But by all means, stamp and destroy it.

phearme• 13 years ago

Congrats! For GNU/Hurd it took years to achieve it.

aortega3• 13 years ago

I really wanted to not sound like an asshole and start critizicing this because I like low-level programming articles but this article is confunsing:

1) You don't need an OS to press keys and make them come out in the screen. An OS is kind of a library that an application uses to do things.

2) I believe what the article refers as "OS" is a small process manager. I don't know if it's preemptive, no mention of timer of any kind.

3) As far as I can see, this is not an OS but a ring-0 application that reads the keyboard using IRQs. IMHO an Operative System should at least supply filesystem/memory manager or process-manager services.

Hacker school alumni, those articles are awesome but please get your concepts right and at least learn the correct terminology or you will only confuse people. But I suspect those articles are vague on purpose to incite controversy.

sp332• 13 years ago

It's not a process manager because there are no processes. And how do you get keypresses to come up on the screen with no OS?

aortega3• 13 years ago

The same way the BIOS makes keypresses come up on the screen with no OS. You don't need an OS to do things in a computer.

exDM694• 13 years ago

You can call it a ring-0 application if wish to argue about semantics or the definition of an OS but you really do end up sounding like an asshole.

Every OS projects starts out as a "ring 0 application" running on bare metal. An application which does nothing except handle interrupts and spew out numbers on the screen to tell that it is working. And most OS projects never really evolve a lot past that, because they are primarily intended as educational tools for the author and nothing else.

If you had a point, you could have made it in a way that makes you seem like a decent person, not an angry antisocial geek.

beering• 13 years ago

Also, aortega wrote "Operative System" instead of "operating system", so because of that incorrect terminology, we can ignore everything in the post. That's how this works, right?

aortega3• 13 years ago

It is not an OS, and telling things that are not true is not educational is delusional. If you people want to remain ignorant and believe things that are not true, be my guest.

moron4hire• 13 years ago

I think, after I clear out my current project log, OS hacking is the next thing on my plate. I'll probably start with drivers, but I eventually want to get into learning the core concepts. OS kernels and compiler design are kind of the last "things" that I've not yet done with programming, and probably couldn't just jump into it feet first and be productive. I've done AI, graphics, real-time embedded programming, high-performance algorithm stuffs, large scale data crunching, etc. etc., and they all pretty much come down to the same stuff: know your math and don't waste cycles. 'Spose that's kernels and compilers, too.

mason55• 13 years ago

When I was in undergrad I wanted to take Operating Systems 2 but it was only offered once/year and didn't match up with my free time. So I did an independent study where my plan was to write a basic ethernet driver for Linux.

I have never been as humbled by any sort of programming in my life. This was back in 2004 so the resources available online weren't nearly as good. I basically figured everything out by reading Intel's x86 documentation from start to finish. It took me a solid 3 weeks just to figure out how to start talking to the ethernet card. I spent days reading the same few pages in the manual over and over trying to understand how to get everything to fit together in assembly & C.

After 9 weeks I was able to address & initialize the ethernet card after the OS booted. I never was able to send or receive any data after a solid 50 hours of work.

Oh yeah, and I got a solid A for my work.

dclusin• 13 years ago

With regards to kernel programming, I'm reminded of the old quote from Star Trek, "It's life, Jim, but not as we know it."

zwieback• 13 years ago

Cool, sounds like so much fun. I remember writing OS/2 drivers and getting interrupt handling wrong would cause strange ways of destabilizing the system. Once you get it sort of going you're afraid every time you press a key or do anything with the system because you know your brand new code is down there in the bowels waiting to cause a triple fault.

After 5 days, my OS doesn't crash when I press a key

Related Stories