BootOS: Monolithic operating system in 512 bytes of x86 machine code

by nanochess • 7 years ago

588 121 7 years ago

kragen30• 7 years ago

For those not aware of the background, the author is a wizard from a secretive underground society of wizards known as the Familia Toledo; he and his family (it is a family) have been designing and building their own computers (and ancillary equipment like reflow ovens) and writing their own operating systems and web browsers for some 40 years now. Unfortunately, they live on the outskirts of Mexico City, not Sunnyvale or Boston, so the public accounts of their achievements have been mostly written by vulgar journalists without even rudimentary knowledge of programming or electronics.

And they have maintained their achievements mostly private, perhaps because whenever they've talked about their details publicly, the commentary has mostly been of the form "This isn't possible" and "This is obviously a fraud" from the sorts of ignorant people who make a living installing virus scanners and pirate copies of Windows and thus imagine themselves to be computer experts. (All of this happened entirely in Spanish, except I think for a small amount which happened in Zapotec, which I don't speak; the family counts the authorship of a Zapotec dictionary among their public achievements.) In particular, they've never published the source or even binary code of their operating systems and web browsers, as far as I know.

This changed a few years back when Óscar Toledo G., the son of the founder (Óscar Toledo E.), won the IOCCC with his Nanochess program: https://en.wikipedia.org/wiki/International_Obfuscated_C_Cod... and four more times as well. His obvious achievements put to rest — at least for me — the uncertainty about whether they were underground genius hackers or merely running some kind of con job. Clearly Óscar Toledo G. is a hacker of the first rank, and we can take his word about the abilities of the rest of his family, even if they do not want to publish their code for public criticism.

I look forward to grokking BootOS in fullness and learning the brilliant tricks contained within! Getting a full CLI and minimalist filesystem into a 512-byte floppy-disk boot sector is no small achievement.

It's unfortunate that, unlike the IOCCC entries, BootOS is not open source.

mysterydip• 7 years ago

How did you learn so much about them? Asking genuinely

kragen30• 7 years ago

I read all the articles on their web site (the ones in Spanish and not Zapotec, anyway), the articles in the press about them, and the comments sections of those articles, and I talked with hacker friends of mine who live in Mexico City. I'm interested in minimalist computing systems like Oberon, Plan9, stage0, MesCC, Scheme, Forth, Squeak, OTCC, the LGP-30, and the Familia Toledo's Biyubi system; myself, I've written things like StoneKnifeForth and httpdito, a web server in under 2000 bytes: https://news.ycombinator.com/item?id=6908064

But only in my dreams do I approach achievements like Biyubi.

sanlyx• 7 years ago

I wondered the same thing, then I did a quick Google search and it turns out they're relatively known in Mexico They also have a web page: www.biyubi.com

ChrisCinelli2• 7 years ago

All the repos in the org looks pretty interesting: https://github.com/nanochess

monocasa• 7 years ago

Yeah, I was all prepared to go into a fairly standard "this isn't an OS, an OS needs to load and execute other programs" before digging into the source, but this totally does that and has a simple shell.

This is ridiculously impressive hacking up there with Woz code.

deathanatos• 7 years ago

I've written a toy bootloader before — a much smaller goal/idea — and my goal with the first sector was to load the second sector!

_xlr2• 7 years ago

I absolutely love that this is not some parody of cyberpunk fiction, but instead reality.

jeremyjh• 7 years ago

Great upsight. This could totally be a Neal Stephenson character.

auslander• 7 years ago

It is cyberpunk reality now. I totally see this tech being sold for premium price to all sorts of security minded people, good and bad.

oh_sigh• 7 years ago

They sounded like a made-up story until I started to look into what you were saying. Why does this family do this? What's the motivation? Pure hacking? Do they run a business in technology and they play around with fun side projects like this?

kragen30• 7 years ago

Why does this family do this?

Why did Mozart write music? What would Mozart have done if he lived in the city of the deaf?

I mean they've written a lot of stuff on their web site about their motivations — they saw that electronics were the wave of the future, they wanted to make Mexico technologically independent and limit the damage of the neoliberal economic model, they wanted to preserve the Zapotec culture from obliteration, and so on — but I think basically it's that they like doing it, in the way Mozart liked music. I'm sympathetic to those other motivations but I don't think they would have been sufficient to maintain their progress in the face of the opposition they've faced from the Mexican government, which continues to adhere to the obsolete "progress is big factories" model mixed with the simply traitorous "progress is installing Microsoft software" model. (They've written about this stuff on their website too.)

As far as I can tell, they make a living by building some industrial control hardware and teaching electronics classes. Maybe a member of the family will comment and offer more detail.

tcbawo• 7 years ago

Since I first saw it posted, I've probably played thousands of games of chess on Nanochess. Impressive achievements, I'd love to learn more of what they've done

djsumdog• 7 years ago

It's unfortunate that, unlike the IOCCC entries, BootOS is not open source.

It's not? What's that asm file in there? It looks like it's the full source code for the image. Or do you mean there's no license file, so it's not free-software/copyright?

Wowfunhappy• 7 years ago

Presumably GP means the latter. Without a license file, the author retains all copyright, you can’t reuse or redistribute any of the code. So it’s source available, not open source.

kragen30• 7 years ago

Update: it's open source now! He's added a BSD license.

edgarvm3• 7 years ago

Oscar Toledo is very talented to write minimal code, but it does mean either his entire family is a society of wizards or they can build a computer or develop browsers, these "achievements" are urban legends without any proof.

Unfortunately, they live on the outskirts of Mexico City, not Sunnyvale or Boston, so the public accounts of their achievements have been mostly written by vulgar journalists without even rudimentary knowledge of programming or electronics. It would be a valid reason in '80s

fao_• 7 years ago

Or you could do a two second google search and find photographic evidence: http://www.biyubi.com/tecnologia.html

aasasd3• 7 years ago

One of the first web browsers in the 90s was written by one dude in four days, with the whole GUI. A browser doesn't necessarily mean CSS and JS.

Sometime in the past couple months, there was a link on HN to a project also by one dude, who's implementing a CPU on breadboards, with wires. (Can probably be found by the score filter from the ‘undocumented features’ of HN: the post got >2000 points.)

Computers aren't necessarily gigantic corporate undertakings.

Arbalest• 7 years ago

Makes me think, are they already the first Archivists. Preserving the manufacture of usable computers when the rest of the support infrastructure for modern tech no longer exists. That might just be me projecting my own thinking onto what you've suggested about this family though.

guiambros• 7 years ago

What an amazing story, thanks for sharing!

I just watched some of TV interviews with the family and saw the rest of his work, and it's pretty amazing. Check out Invaders in 512 bytes [1][2], or Aardvark game for the Atari 2600 ([3]).

Also seems he has just self-published a book (as of July 27):

Do you would like more details on the inner workings? This program is fully commented in my new book Programming Boot Sector Games and you'll also find a 8086/8088 crash course! Now available from Lulu [4]

[1] https://github.com/nanochess/Invaders

[2] https://www.youtube.com/watch?v=pC976eejfms

[3] https://www.youtube.com/watch?v=HNZfw6jShng

[4] http://www.lulu.com/shop/oscar-toledo-gutierrez/programming-...

tempguy9999• 7 years ago

This might interest me as someone whose job is the absolute other end of the spectrum, in databases and HLLs. Reason is that I'm concerned with CPU performance and that largely depends on caches (after you've done any algorithmic improvements).

The unqualified claim - may I say lie - that CPU time is cheaper than programmer time was never true and never will be[0] because it depends, it depends on the economics, and a surprising amount of my time has been taken up optimising for speed on multicore modern CPUs. Exactly what other people say isn't worth doing, right up until it is - and it often is.

Optimising for speed often means optimising for caches, and that means optimising for size[1], so I'm going to dive into this guy's stuff.

[0] and I'd ask people to stop repeating it unless the larger context is accounted for.

[1] and cache-friendly access patterns; another issue.

qx89l4• 7 years ago

Also seems he has just self-published a book (as of July 27):
Do you would like more details on the inner workings? This program is fully commented in my new book Programming Boot Sector Games and you'll also find a 8086/8088 crash course!

I created one such game [1] a few years ago and had plenty of fun. I'm pretty shocked to see an actual book published specifically about this niche topic. I haven't read it, but I hope it gets some recognition, because IMO toying with 8086 is one of the best ways to get into bare-metal coding.

[1] https://github.com/qx89l4/utris

msla3• 7 years ago

You might want to update this one: Mexico City isn't a backwater anymore. Try... somewhere in Afghanistan, maybe, or South Sudan.

kragen30• 7 years ago

Listen, UNAM is one of Latin America's greatest universities, and Mexico City is highly industrialized, but Mexico City is like Milwaukee or Houston or Albuquerque or Buenos Aires, not like Sunnyvale or Boston or Singapore or Shenzhen. If you're a hacker in Mexico City, you're gonna be pretty isolated by comparison to a hacker in a tech hub, and you're swimming against the tide of a corrupt and technically incompetent government in the same way you would be in Houston, a way that you won't be in Sunnyvale or Boston.

I'm well aware that people in Mexico City aren't illiterate paupers. That's not what I was saying, man. But you're going to have a hell of a time launching a new kind of laptop in Mexico City, and that's what they were trying to do.

vanderZwan• 7 years ago

I wonder how few bytes he'd need for a brainfuck implementation

wgx• 7 years ago

Also by Óscar Toledo, chess in 1k of JS: https://js1k.com/2010-first/details/750

petee• 7 years ago

He added a BSD License -- https://github.com/nanochess/bootOS/blob/master/LICENSE

kragen30• 7 years ago

Oh, that's wonderful news! I had assumed the lack of license was intentional.

little_goat_boy• 7 years ago

Do you have any suggestions on where to find any english language documentation about the family? I am ravenously curious.

kragen30• 7 years ago

They have some English pages on their website, although they haven't been updated in several years: http://www.biyubi.com/eng_principal.html

If you make fun of the errors in the translation I will make fun of you mercilessly for not knowing any Spanish at all.

kragen30• 7 years ago

Update: it's open source now! As of two hours ago! He also open-sourced bootBASIC, but not fbird, pillman, or invaders. (I had assumed that the lack of any license was intentional, perhaps to keep other people from selling books based on his work, in competition with his own.)

Unfortunately I can't update my top-level comment above.

gwern• 7 years ago

If you have the time, it would be a great project to translate and summarize all the relevant material about the Familia Toledo.

kragen30• 7 years ago

Hmm, I hadn't thought about that! I wonder if they'd be willing — a lot of their website is under a somewhat vague quasi-open-source license, but they might be chary of "helpful" outsiders who want to "help", as I'm inferring they've had some bad experiences in the past.

They've translated some of their pages themselves, although the translation is somewhat out of date: http://www.biyubi.com/eng_principal.html

msla3• 7 years ago

For those not aware of the background, the author is a wizard from a secretive underground society of wizards known as the Familia Toledo

Wow. None of this is plausible.

ChrisCinelli2• 7 years ago

It seems that this more than a OS is a minimal file system with a obvious limited feature set that relay on the https://en.wikipedia.org/wiki/INT_13H - It assumes that a BIOS that contains a lot more machine code is available.

For the new generations this will seem not to have any real applications beyond code golfing.

But from a nostalgic point of view, it brings me back to the demoscene era. I still fell a wistful longing for the old days thinking to fr-041:debris ( https://www.youtube.com/watch?v=tq2llbP52Zo ) 7:20 minutes of awesomeness in less than 199 Kb.

BTW, I just google it and apparently there is plenty of events going on... Coming very soon: https://www.assembly.org/summer19/

kragen30• 7 years ago

A filesystem, a shell, a program loader, and a hex editor! Well, maybe "editor" is overselling it. But from my point of view, it's as much an "OS" as MS-DOS was. MS-DOS also depended on the ROM BIOS.

I agree that it's probably not practically useful. But it's a brilliant hack!

devnulloverflow3• 7 years ago

Yes, BootOS very much in the same space as MS-DOS, although leaning even further on the "less-is-more" axis. It does the same basic things (though I suppose it doesn't do the BASIC thing).

Interestingly, there is modern OS in the same space. That's UEFI. Unti I played with it a few months ago, I thought it was some mysterious firmware-adjacent black-box. But actually it's just an OS, very similar to DOS (though it runs in protected mode).

It has a shell and some basic built-in commands including text editors and such. And most importantly it lets you run executables that a stored on a FAT (well, FAT32) filesystem.

The name BootOS could very well describe UEFI. And equally, a slightly more capable version of BootOS could serve as (very underpowered) UEFI replacement.

caf2• 7 years ago

For me the nostalgia this brought back was a memory of boot sector viruses, particularly the way the 'format' command propagates itself onto the new disk.

userbinator• 7 years ago

The significance of 512 bytes is that it is the standard sector size for a PC storage device (the size seen by software; newer hard drives have 4KB physical sectors, but can still be addressed as if they had 512B "logical" sectors), and a traditional BIOS reads the first sector of the boot device into memory and jumps to it in order to begin the OS part of the boot process. A normal OS bootsector would then find and load the next part of the OS, which may involve things like understanding FAT or NTFS filesystems; here's a (rather enthusiastic) analysis of many boot sectors from different versions of DOS and Windows:

https://thestarman.pcministry.com/asm/mbr/index.html

Writing a tiny program that lets you modify memory, and more importantly execute it, is the first step in the "bootstrap pilgrimage" to creating your own environment and eventually OS, but this one seems deliberately limited; nonetheless, starting with this, one can use it to create the next stage of the bootstrap, 512 bytes at a time.

non-entity• 7 years ago

Wow this is really cool. I honestly expected this to be just a program that would boot to some text output and maybe a very simple shell, but it looks like this more than that.

bArray6• 7 years ago

I have also developed a very small kernel in 512 bytes, it's a fun exercise [1] (this is a really old project that was uploaded to BitBucket long after it initially started). I quite like this BootOS project, you can of course see the compromises that any OS this size has to make.

I would make a recommendation to the BootOS project which is to squeeze a function table as I have done in order to allow programs to call functions in the kernel. If not, you either end up re-writing these functions for each of your programs, you have to recompile your programs any time you change your kernel or you end up trying to maintain jump positions.

For those still reading, I'm working on a large overhaul of saxos which includes very simple multi-tasking (currently working). The intention will be to have the kernel load a 512 byte GUI on top and then for the programs to have access to a terminal within the GUI. Once that's working I then want to move towards also allowing programs to draw to the Window, but that also brings its own issues.

[1] https://bitbucket.org/danielbarry/saxoperatingsystem/src/mas...

kragen30• 7 years ago

Oh awesome! This one also has a tiny filesystem and prompt, but no UI for typing machine code in, is that right? But the filesystem supports more than one sector per file? Saxos is open-source, unlike BootOS, even if maybe that would be clearer if you used a widely-recognized license.

I don't think BootOS tries to export its filesystem API to user programs. BootBASIC in particular doesn't seem to have a way to save and load programs.

Brilliant work!

(Apparently I heard about this before and never looked at it? Because I have a clone of it on my laptop dating from September 14th.)

bArray6• 7 years ago

Oh awesome! This one also has a tiny filesystem and prompt, but no UI for typing machine code in, is that right?

Yes, but it offers more to the programs that it loads. There's a trade-off when you go this low on disk space!

But the filesystem supports more than one sector per file?

SaxOS supports half the disk, but each file has to be 512 bytes [1]. There are potentially work arounds for this limitation of one sector per file, but it's just hacky. (For example, you could name your files txt0, txt1, ... - or you could leave some header information in your program's file for more data.) I will fix this in a future version though, the plan is to keep loading sectors by the same name in the file table.

Saxos is open-source, unlike BootOS, even if maybe that would be clearer if you used a widely-recognized license.

When I wrote this I had no idea about licenses :) I will look to update this, but the project is mostly just archived for now.

I don't think BootOS tries to export its filesystem API to user programs. BootBASIC in particular doesn't seem to have a way to save and load programs.

It takes very little to extend the functionality of your kernel to your implementing programs, hopefully BootOS will do this in the future. At some point I want to write a simple self-writing documentation (using the comments structure) so that I can always produce up-to-date documentation for the OS.

Brilliant work!

Thanks!

I was trying to build a very small low end compression algorithm for it to, but unfortunately I came hard up against entropy. The idea worked (as I envisioned it) but it had the effect of just transforming the dimension the data was stored in and adding unthinkable amounts of computational power and memory being required to get it back.

(Apparently I heard about this before and never looked at it? Because I have a clone of it on my laptop dating from September 14th.)

Awesome! My plan for quite a while has been to write a small book and guide people through building their own very simple OS. I think people building their own 512 byte boot-able kernel is a nice intro into OSes whilst also having enough limitations that at some point you can say "this is complete". Anyway, it's what I wish had existed when I first set out may years ago!

Speaking of which, I must give a shout out to MikeOS which is where I learned about assembly and OSes [2].

[1] https://bitbucket.org/danielbarry/saxoperatingsystem/src/96c...

[2] http://mikeos.sourceforge.net/

nanochess3• 7 years ago

Cool! I didn't knew of your work. I've been working in the service table, just that it isn't ready yet because I want DS to handle filenames and ES the buffer but currently the segment mangling still is destroying the directory. Almost ready ;)

nanochess3• 7 years ago

bootOS now has a service table and an example program counter.asm to demonstrate it :)

kazz0302• 7 years ago

Pretty cool!

Eun• 7 years ago

What I am still looking for, is for a boot os that can boot a normal .iso/.img file from its own device. This will be the perfect usb boot stick.

1. Flash boot os on the boot sector, place any iso/img file on the stick.

2. Boot from the stick

3. select the iso you want to boot

Don't know it this is possible though... (I know that there is a emulation mode for virtual cd drives in the bios but thats about it...)

ComputerGuru• 7 years ago

EasyBCD can give you that, but you must run it to update the boot code each time you add an ISO (ie no dynamic detection).

wolfspider• 7 years ago

It is possible with EFI based boot loaders from what I've seen. I was looking for something similar and ended up discovering rEFIt and rEFInd. Although it isn't technically an OS it can autodetect entries from the MBR. http://www.rodsbooks.com/refind/

arantius• 7 years ago

You're looking for YUMI: https://www.pendrivelinux.com/yumi-multiboot-usb-creator/

It's not 100% compatible with every ISO, but it's definitely possible to boot more than one thing from a single USB drive.

notpeter• 7 years ago

I've seen various software that does this with multiple installer/liveCD ISOs for Linux distributions but none that could actually boot all arbitrary ISOs (Windows installers, DOS rescue disks) which need to be on a logical device partition.

I used to have a 2.5 enclosure that supports this in hardware (Zalman ZM-VE300) but selection of ISO is done on-device before booting. I don't see any reason you couldn't build a hardware/software combo where selection on-screen made the USB hardware mount the selected ISO as a new logical device and then booted it.

[1] http://www.zalman.com/contents/products/view.html?no=212

t3x4c0• 7 years ago

BootOS developed by a 10x Rockstar Ninja google dev! Oh no, wait...it's a mexican who works with his family at "Familia Toledo"... Guys, i think perhaps we got it all wrong.

BootOS: Monolithic operating system in 512 bytes of x86 machine code

Related Stories