Oasis: Small statically-linked Linux system

https://github.com/oasislinux/oasis

by ingve • 4 years ago

381 219 4 years ago

github.com

snvzz2• 4 years ago

and is probably better compared to a BSD.

That's a very unfortunate comment in the second paragraph.

Sakos2• 4 years ago

Should probably clarify in what ways (or for what) it's supposedly better. That's quite the inflammatory statement.

camgunz• 4 years ago

I think the meaning here is "better <to be> compared to a BSD", as in it's closer to a BSD than a Linux in terms of design and ethos.

projektfu2• 4 years ago

I don't read that sentence to mean it's better than BSD, rather that Oasis is more like BSD than a typical Linux distribution.

"It is better compared to" is an idiom meaning "the comparison is more apt to".

rafram• 4 years ago

You’re misreading it. The author means that it’s more similar to a BSD than to a typical Linux system, as the previous sentence makes clear.

unsafecast2• 4 years ago

((better compared) (to BSD)), not (better (compared to BSD)) :)

endorphine• 4 years ago

Or:

"[...] is better compared to BSD", not "[...] is better, compared to BSD".

speed_spread2• 4 years ago

phrasing should be changed to "similar to BSD in design".

mforney2• 4 years ago

Sorry for the confusing wording. I meant that oasis has more similarities to a BSD than a typical Linux distribution.

I've updated the README to be a bit clearer.

snvzz2• 4 years ago

Good work.

mmargerum• 4 years ago

Now if I could just boot this from a cartridge I'd approach the last time programming was truly joyful for me.

projektfu2• 4 years ago

I played around with the Qemu image and I really liked it. I wasn't even aware of some of the software they chose, like Velox.

KSPAtlas• 4 years ago

I used this before, pretty impressive how much it can pull off

dijit7• 4 years ago

I ran a really non-scientific test before on startup speed between Oasis vs a slimmed down direct kernel booted Debian with systemd.

Obviously I'm writing this because Oasis absolutely trounced it.

So; Oasis is currently my go-to for small deployed VPS's.

(here's the video if you actually care: https://www.youtube.com/watch?v=X2Aw5SSqYFo you'll note that the systemd one was delayed by the network on start- some people are going to argue that this is an artificial delay, but this is the truth of a startup and oasis had already booted before the delay anyway)

zamadatix2• 4 years ago

FWIW I get a faster boot time than that slimmed Debian VM with a physical Celeron class physical box booting a stock kernel, even including the real BIOS and hardware initialization time. I don't think systemd really has much to do with why that VM boots slow.

dijit7• 4 years ago

I don’t think that’s true.

That machine is a threadripper 3970x with 256G 3.2GHz DDR4, with a 4xNVMe RAID0; it’s a disturbingly fast glass-cannon.

I mean, it’s not an accurate test for sure; but I seriously doubt your VMs can boot faster stock and with the bios, on a slower machine.

raggi2• 4 years ago

Debian is why debian boots slow.

LtWorf3• 4 years ago

Normally for me on debian, just compiling the kernel to remove most modules and get rid of initrd gave great results. But the kernel config is not the same on all the machines of course.

itvision2• 4 years ago

I don't know man.

I see a display manager (lxdm under Xorg) password prompt 3 seconds after the GRUB menu.

And that includes quite a lot of stuff:

systemd-analyze blame | wc -l

herpderperator2• 4 years ago

If you want super fast boot, go with Arch Linux.

herpderperator2• 4 years ago

Why am I being downvoted? Is it because I didn't show proof? Here's my video showing Arch Linux (systemd) booting to the login prompt in 3 seconds, including network: https://www.dropbox.com/s/hgeopuou9rzq6ru/linux_vm_on_macos....

dataflow• 4 years ago

How long did it take Oasis boot to have network interfaces ready?

dijit7• 4 years ago

They were ready before it hit the login prompt;

hulitu2• 4 years ago

Minimal bootstrap dependencies. Any POSIX system with git, lua, curl, a sha256 utility, standard compression utilities, and an x86_64-linux-musl cross compiler can be used to bootstrap oasis.

So you need a custom POSIX system to bootstrap oasis.

All software in the base system is linked statically,

this could be a good thing, however

including the display server (velox) and web browser (netsurf).

No X server and an obscure web browser. Useless.

sophacles2• 4 years ago

Name me an OS that doesn't need an OS to build it.

hulitu2• 4 years ago

It needs a very special OS: "git, lua, curl, a sha256 utility, standard compression utilities, and an x86_64-linux-musl cross compiler"

80h• 4 years ago

No X server and an obscure web browser. Useless.

Indeed, it uses a custom Wayland compositor. To me it looks like using a minimal Wayland compositor would fit the project better than using X. The same could be said about using Netsurf.

castlec• 4 years ago

um, it says if you have the libs and configured compiler, you can build it.

there's also nothing stopping you from doing static builds of whatever application you want to include in your OS.

sleepydog• 4 years ago

Velox appears to be a Wayland compositor, and supports Xwayland.

waynecochran9• 4 years ago

Death to shared libraries. The headaches they cause are just not worth the benefit. I have been dreaming of someone cresting a system devoid of them.

speed_spread2• 4 years ago

You're just embracing another set of problems. Shared libraries were introduced for a reason. The are complex to handle, but that's a long standing tooling problem rather than an inherent issue of the model. I think we're actually just getting the hang of it with stuff like Nix and containers.

shrimp_emoji• 4 years ago

Shared libraries were introduced for a reason.

Lack of storage and bandwidth in the 1970s

arccy• 4 years ago

nix and containers actually eliminate all the benefits of shared libraries while still having all their problems

rattlesnakedave3• 4 years ago

“Shared libraries were introduced for a reason.”

I’m pretty sure the reason is “X11 was large and badly factored, and people who should have known better thought it would save disk space.”

The problems they claim to solve are updates across applications at once, but in the enterprise how often does that actually happen? Application authors still have to test against the new version, and sysadmins still have to roll out the changes required to support the update. In practice they may as well statically link.

Let’s not forget the problems they introduce are significant. Implementations are complex. Applications become less portable. ABI versioning is a pain (versioned symbols are gross). Semantic changes in the shared lib cause chaos. Exec is slower because of the linking done at load time. Dynamic libs are bigger than static ones, AND they’re loaded into memory. More difficult to verify the correctness of a given executable.

The alleged “benefit” here really does not outweigh the cost. If we were doing static linking from the beginning, it’s difficult to say if we would have wound up with things like docker or nix in the first place.

alkonaut3• 4 years ago

Yes. But the other set of problems exists anyway. Even if you drop in a new system version of a shared library, you still need to consider what other versions are used apart from the system library. Any app (other than those provided by a system package manager of course) can have a local version different from the system version and any app can have linked it statically?

flohofwoe2• 4 years ago

Shared libraries were introduced for a reason.

Those reasons were floppy disks and RAM that was measured in kilobytes though.

onlyrealcuzzo• 4 years ago

What are the headaches with shared libraries?

Assuming that you pin the version of the library you need, two applications using the same library /shouldn't/ cause issues.

In most *nix systems - each instance of a shared library has its own data.

rattlesnakedave3• 4 years ago

“ Assuming that you pin the version of the library you need, two applications using the same library /shouldn't/ cause issues.”

Then why bother?

waynecochran9• 4 years ago

I am assuming you know the answer to the question if you have been developing code for awhile. What makes the problem worse is that it is transitive. I link against Colmap libs, which links against Ceres libs, which links against OpenMP, … these are ever changing code bases and I have to create executables that run on local machines, Azure cloud machines, AWS cloud machines, … chafed down shared lib mismatches becomes a nightmare.

ksec• 4 years ago

I dont disagree. But wondering if I am missing anything other than the usual cons such as Memory usage, code ( Storage ) size and longer compile time.

waynecochran9• 4 years ago

I understand the motivation for shared libs is the reduced memory footprint for large common libs. I have an alternative idea that avoids the headaches of shared lib mismatches and still can reduce the memory footprint. Have relocatable code be statically linked but stored with a hash. When loading a binary see if common relocatable code is already loaded .. if it has the same hash then it is exactly the same and can be safely shared. No version mismatch headaches.

hlsnqp2• 4 years ago

So next time a library such as OpenSSL has a critical security vulnerability that needs to be patched, you'd rather every single binary that has it statically linked is updated, rather than a single shared library file?

rattlesnakedave3• 4 years ago

Google “OpenSSL dependency conflict” and view the results. There is significantly more work going on when OpenSSL releases and update than “just update the single shared library file.” Applications often need to be updated as well, so you may as well just statically link the new binary.

Ericson2314• 4 years ago

If you keep around the .o / .a files this isn't so bad.

ori_b2• 4 years ago

Sure. Networks are fast enough.

alkonaut3• 4 years ago

This argument comes up every time this discussion comes up (on flatpaks etc). And for almost all systems I’d say “yes, absolutely”.

One headache over another.

What’s to guarantee applications haven’t statically linked a library (in whatever version) rather than linked a “system” version of a library?

With the dynamic one you have both problems…

waynecochran9• 4 years ago

Yes. I can think of worse problems w shared libs. Should we start a list?

kortex• 4 years ago

In the majority of cases: Absolutely. That is basically the direction things have gone with "just docker build the image and redeploy".

The amount of frustration dynamic linking causes is just not worth it in today's day.

I think that calculus looks very different 20, 10, maybe even 5 years ago, when getting packages rebuilt and distributed from a zillion different vendors was much harder.

flohofwoe2• 4 years ago

Considering the complexity that shared libraries add over static linking: yes absolutely.

glowingly• 4 years ago

We just didn't update Log4j since it broke a critical, old, closed source offline program. Vendor has long dropped support for this old program. Cue that up to the usual executive missteps that we get from up high.

Yay, we get to keep on using old Log4j now because one program holds it back.

enriquto• 4 years ago

you'd rather every single binary that has it statically linked is updated, rather than a single shared library file?

Yes! A thousand times yes. I'd rather reinstall every single compromised program than deal with the ridiculous complexity of dynamic linking.

worldshit• 4 years ago

you only need to relink, not recompile. i think oasis caches artifacts so there's no overhead.

indy• 4 years ago

Yes

marssaxman• 4 years ago

Oh, yes, I certainly would. Shared libraries are an expensive, trouble-prone solution for that relatively uncommon problem.

Anyway that's not why we have shared libraries. They were invented so that memory managers could load the same library once, then map it into the address space for every process which used it. This kind of savings mattered back when RAM was expensive and executable code represented a meaningful share of the total memory a program might use. Of course that was a very long time ago now.

encryptluks22• 4 years ago

I used to feel the same way. In an ideal world all packages would be automated and developers would have a way to alert packaging tools that a new version is release, so that it can be run through CI/CD and users would get new packages almost instantaneously without any breaking changes.

However, until that happens there is nothing wrong with shared libraries. They exist for good reason. They save bandwidth and ensure that you're not getting old libraries.

waynecochran9• 4 years ago

The save bandwidth? Do you mean memory? You want to link against the exact same code you developed and tested your program with. Ensuing you get and exact match is critical for robust code.

icedchai2• 4 years ago

You could try DEC Ultrix. (Oh, did you mean a modern system?)

benreesman16• 4 years ago

When it comes to mandatory dynamic linking, it's always a Beautiful Day for an auto-da-fé: death is too good for mandatory `glibc` and `libstdc++`.

As for a system devoid of it, I have grand plans, but for now: people do it with Docker every day! That's what Docker is. It's not broadly technically feasible right now to get a low-friction static link on a mainstream distro with mainstream libraries, but life, erm, technology Finds a Way. For now that's Docker, but the modest </s> enthusiasm around it shows that people want a way out pretty fucking bad and there will be better options soon.

arinlen9• 4 years ago

As for a system devoid of it, I have grand plans, but for now: people do it with Docker every day! That's what Docker is.

No, Docker really isn't what you think it is.

Docker isolates processes, and provides a convenient platform independent way to deliver stand alone applications, complete with software defined networking and file system abstraction, at the expense of shipping the whole tech stack, including everything plus the kitchen sink like full blown web servers.

Thus a .DEB package which otherwise would be 10MB ends up being a Docker image often weighing over 100MB.

And pray tell what's Docker's solution to software updates? Well, it's periodic builds that force downstream dependencies and users to re-download payloads of well over 100MB a pop.

And you advocate this as an alternative to shared libraries?

Who in their ri

arinlen9• 4 years ago

Death to shared libraries. The headaches they cause are just not worth the benefit.

Completely disagree. Even though one size does not fit all, anyone who makes sweeping statements about static libraries is just stating to the world how they are completely oblivious regarding basic software maintenance problems such as tracking which software package is updated, specially those who are not kept up to date on a daily basis.

nicoburns• 4 years ago

If there was a standard for specifying which static library a binary has linked, this would be a non-issue.

waynecochran9• 4 years ago

I like the old school days of just building everything from source. Similarly the Linux Kernel was statically built -- before modules. I knew the whole system from top to bottom.

mariusor• 4 years ago

I'd say this is a solved problem. It's easy to get all build dependencies of a package and for every library that gets updated you build the world.

It's a more computing power and bandwidth that is required, but the maintenance burden is mostly solved by now by modern package managers or tooling on top of not so modern ones.

benreesman16• 4 years ago

Dynamic linking by default has been vocally criticized by everyone from Carmack to Bell Labs/Plan9/golang people. Many if not most of the big scale companies have historically avoided it like the plague. It has horrific costs in performance, security, reproducibility, and ultimately comprehensibility of any system.

People pay out the ass in droves via Docker (huge-ass images, clunky composability, cgroup/namespace complexity) and Nix (horror-movie learning curve) and I'm sure other things to get around it / emulate static linking. `sudo apt-whatever upgrade` and pray is pretty much DOA in even low-complexity settings. People are voting with their feet.

You can disagree, and if you're not affiliated with Debian or GNU I might even believe that you come by that opinion honestly, but this gigantic bloc of people who hate it are not all "oblivious".

paskozdilar• 4 years ago

Dynamic linking is just poor man's IPC.

SkyMarshal• 4 years ago

I have been dreaming of someone cresting a system devoid of them.

Doesn’t that pretty much already exist now with immutable system builds, eg Fedora Silverblue, GUIX, NixOS? Each package pulls in its own dependencies with the specified version and config, and then no changes can be made once the system is built. Only way to update is to rebuild the system with the updated versions. Costs more disk space but that is cheap these days, and solves shared dependency hell.

goodpoint6• 4 years ago

Absolutely not. Dynamic libraries are crucial for security.

Static linking again and again has proved to be a security nightmare. It makes security updates dependent from upstream release or full rebuilds making things difficult.

spc476• 4 years ago

But aren't you dependent from upstream anyway? Or do you recompile from source the broken dynamic library? In my experience, most sysadmins these days don't compile from source.

naasking4• 4 years ago

Proved to whom, when, and in what context? Systems that were designed for easy reproducible builds from the ground up are fairly new, so how sure are you the context hasn't changed?

SuperSandro20002• 4 years ago

Which is not feasible for misters like qt WebKit or chromium.

its_bbq• 4 years ago

Is this meant more to be used on a small embedded Linux system? As a Serverless OS for fast startup?

mro_name2• 4 years ago

The core values are very appealing to me.

what is the typical update routine a la 'sudo apt-get update && sudo apt-get dist-upgrade'?

bragr• 4 years ago

From the readme:

  No package manager.
  Instead, you configure a set of specifications of what files from which packages to include on your system, and the build system writes the resulting filesystem tree into a git repository. This can then be merged into /, or pulled from another machine.

If you're looking for a package manager, you're in the wrong place.

mforney2• 4 years ago

git -C /src/oasis pull && samu commit && doas git -C / merge

The first command pulls updates from the source repository, the second command builds a new filesystem tree and commits it, the third command merges that commit into your / repository.

mro_name2• 4 years ago

thanks for oasis!

tyingq• 4 years ago

I'm curious how it handles the client side dns resolver. That's one of the sticking points for "fully static linking".

Edit: Looks like it uses musl's resolver. Which is better than I remember it when using it a long time ago. It does still lack some things like IDN/punycode support. And things like this: https://twitter.com/richfelker/status/994629795551031296?lan...

Pluggable authentication modules (PAM) may also be an issue with this setup, probably at least requires a lot of manual research and twiddling.

suprjami• 4 years ago

The musl resolver also lacks "no-aaaa" which glibc grew a week or two ago.

gravypod2• 4 years ago

I'd love to see someone port this to Bazel and build a ruleset to generate os images for KVM/VirtualBox/flashing to bare metal. This would be great for having a generic platform for building embedded systems from source.

benreesman16• 4 years ago

I'm working on a Nix/NixOS based polyglot Bazel build infrastructure (that we're using in production) and I've almost got `glibc` and `libstdc++` squeezed all the way out. They're like fucking Kudzu (which is not an accident), but I'll get 'em eventually. Once `glibc` and `libstdc++` are all the way gone it will be a hop, skip, and a jump to doing what you're describing.

I anticipate it being ready for open source as a standalone flake by the end of 2022.

dijit7• 4 years ago

I literally had that conversation in their IRC channel and the resounding consensus was that “why would you want that”

I was asking if anybody had done it yet, but since there’s people who want that I will submit some BUILD field to oasis. :)

jrexilius• 4 years ago

This line is a bit concerning to me: "and libtls-bearssl, an alternative implementation of libtls based on BearSSL" It sounds like a custom rolled variant of a low-adoption SSL lib that isn't itself very well tested.. It may be better.. it may be horribly flawed or backdoored in a non-obvious way. But I also get how horrible OpenSSL is..

synergy20• 4 years ago

same here, in fact the last bearssl release was 4 years ago, very inactive if still alive.

mbedtls might be better here, but it does not have tls1.3 fully ready(I don't think bearssl has 1.3 either).

wolfssl could be the best, small and robust with new features, it is dual licensed if you want to use it in commercial releases.

badrabbit• 4 years ago

Debian has reproducable builds going for it. I would be very interested in static gentoo like source distro though.

numlock86• 4 years ago

BearSSL is incredibly small and well written, but is not widely adopted.

Well, probably because the author himself discourages you from using it in anything serious. Crypto libraries need a lot of testing and reviews by experts, which BearSSL doesn't have yet. Thomas Pornin is an incredibly smart guy when it comes to crypto and BearSSL is my crypto library of choice for small (mostly embedded) projects, too. Projects like this might help to get it the attention it needs.

intelVISA2• 4 years ago

I use BearSSL in a few toy projects, would love to see it get more traction and eventually some proper cryptographic audit if it moves out of alpha.

benreesman16• 4 years ago

“I tend to think the drawbacks of dynamic linking outweigh the advantages for many (most?) applications.” – John Carmack

<rant>

I don't like to be a parrot for the Plan9 people [1] (they've been winning their own fights for a while now), but they've assembled a better set of citations than I could in the time I'm willing to devote to an HN comment.

Dynamic linking is fine, in very specific, very limited cases. Python extensions. Maybe you want to do a carve-out for like, some small hardened core of an SSL implementation or a browser because you've got actual hot-deploy in your (massive) installation/infrastructure.

As a basically mandatory default it's a chemical fire next to an elementary school. It started gaining serious traction as a horrific hack to get a not-remotely-finished X Windows to run on Sun boxes in like, +/- 1990. It wasn't long before the FSF had figured out that they could bio-weaponize it against interoperability. The `glibc` docs are laughing in their beer while they explain why static linking on "GNU" systems is impossible, and also makes you a criminal. They don't even pretend to make a cogent argument (they said the quiet part loud on the mailing lists). It's to kill other `libc` and `libc++` (fuck you small-but-vocal anti-interoperability FSF people!). Don't want LLVM to finish you off? Start interoperating better. There are a lot of us who think "Free Software" has to mean that I can use it in legal ways that you philosophically disagree with, and if it happens to be better than your Win32-style lock-in? Well that's just groovy.

Dynamic linking by (mandatory) default: Just Say No.

Myths:

- People want it: Docker. QED.

- Secure: Dynamic linking is the source of tons of security vulnerabilities. More generally, the best way to not trust your system is to not be able to audit the literal code you're running without jumping through insane, sometimes practically impossible hoops. If you want "father knows best" security, you need self-patch like Chromium and shit do now. And if you're willing to trade performance for security by the gross, flat pack or something.

- Performant: Dynamic linking forces address patch-ups, indirect calls, and defeats many optimizations. It thrashes TLB. It slows startup of even simple programs.

- Necessary: Text pages are shared. I don't care if you've got 9, 90, or 90k processes using `printf`: it's going to be in the same text page unless something else went badly, badly wrong.

</rant>

[1] http://harmful.cat-v.org/software/dynamic-linking/

medoc• 4 years ago

Your last point is a bit off. Text pages are shared iff they come from the same file. So your point works if the 90k processes are the same program. Else: shared libs...

benreesman16• 4 years ago

Yeah I was typing too fast in ranty mode, thanks for calling it. I did a much more detailed answer that gets it (mostly) right replying to another commenter: https://news.ycombinator.com/item?id=32461766

int_19h2• 4 years ago

The `glibc` docs are laughing in their beer while they explain why static linking on "GNU" systems is impossible, and also makes you a criminal.

Can you provide a reference? I skimmed through the glibc manual but couldn't find anything like this.

It's to kill other `libc` and `libc++`

How does this follow from dynamic linking?

benreesman16• 4 years ago

Good primary sources are the `glibc` documentation, especially the FAQ [1], and the `musl` docs, especially the "differences from glibc" section [2] and the FAQ [3], which is basically a long list of ways that `glibc`/`gnulib` developers have leaned into ever-more-broken/nonstandard behavior. If you dig through the GNU mailing lists (which is where I draw the line for an HN comments), you'll find megabytes of admission that this is in an effort to make "GNU/Linux" (or as I've taken to calling it, GNU "plus" Linux, Linux being merely the kernel...), to quote Fassbender as Steve Jobs: "Closed system! Completely incompatible with anything!"

But the short story is that both NSS (DNS lookup), and `iconv` (locale handling) don't work without `glibc`-specific dynamic linking shit. Ostensibly this is because of things like merely being able to read decades-obsolete multibyte character encodings, and not able to write them, is a bridge too far to support at all, let alone as a default. They've also got a bridge they'd like to sell you (see mailing lists).

It's just 90s Microsoft vendor lock-in: "embrace, extend, extinguish" we used to call it. Stallman was a hero for getting the free software movement rolling in the 80s. But in this business, you either die a hero or live long enough to become the villain, and Bill Gates himself would be proud.

[1] https://sourceware.org/glibc/wiki/FAQ [2] https://wiki.musl-libc.org/functional-differences-from-glibc... [3] https://wiki.musl-libc.org/faq.html

jancsika• 4 years ago

Would dlopen be disallowed on this system?

E.g., technically speaking most (all?) audio plug-ins are shared libs...

denton-scratch• 4 years ago

Sounded good to me, until I found out there's no repository; you just pull everything from github. Then I stopped reading. What could go wrong?

rishav_sharan• 4 years ago

As someone who clearly doesn't understands this well enough - can I run a desktop environment on top of it?

pas• 4 years ago

it has Velox which is a Wayland compositor, so you can start running static linked Wayland programs.

and of course you can add things like Firefox, LibreOffice or whatever, but you either need to spend many many cursed nights and dark days to get them to compile statically ... or just add a dynamic linker (ld) and add the dependencies... but at that point you built a second independent system on the same filesystem.

gkhartman• 4 years ago

I was ready to give this a try, but the use of BearSSL is concerning. It goes on to say that it's not a widely adopted SSL implementation, so how can it be trusted? I suppose adoption has to start somewhere, but I'll watch from a safe distance.

baby• 4 years ago

I used to audit tls libraries for a living and bearssl is the library I would choose today if I had to make a choice

ComplexSystems• 4 years ago

Can you elaborate?

logdahl• 4 years ago

The discussions on static- vs dyamic linking piqued my interest. Is there any resources I should read about the history and concrete pros & cons of linking techniques?

ttgurney• 4 years ago

I'm really grateful for Michael Forney's work on this project since I found it a good resource for getting a variety of popular packages to build statically.

I ended up starting my own statically-linked Linux distro, but with some creative differences like shell scripts (vs Lua scripts) and package management. I recently "finished" it (close enough anyway) to the point that I run it as my daily driver.

Not trying to plug my project (it's not public, not yet anyway) but if anyone wants to discuss this sort of thing (distro creation and static linking in particular) and compare notes via email, I'd be happy to.

lr1970• 4 years ago

Interesting project but according to the github history it has had only one official release in 2017 and there has been only 34 commits so far this year. Is the project in low maintenance mode?

Oasis: Small statically-linked Linux system

Related Stories