Linux compatibility in IncludeOS

https://www.includeos.org/blog/2018/musl.html

by ingve • 8 years ago

85 34 8 years ago

freedomben• 8 years ago

I'm pretty ignorant on Unikernels so maybe there's another solution, but not being able to `fork()` is a significant problem for many applications.

__bjoernd• 8 years ago

My understanding is that unikernels are single-application by design. I agree that a mechanism to spawn other apps is useful, but in this case it won't be a simple fork but the creation of a complete new os instance.

tyingq• 8 years ago

I suspect GP is referring to single applications that aren't single process. Postgres is one example. Vacuum and checkpoint are separate processes.

Applications also sometimes have separate processes for sentinal functions, logging, etc. Or default configurations that are multi process, like many load balancers, web servers, API gateways, etc.

chris_wot2• 8 years ago

There are plenty of applications that used multiple processes.

perbu8• 8 years ago

On Unix you typically fork as a part of initializing your application in order to demonize. Or when you are executing another program so a fork/exec.

The first one isn't needed in a unikernel. You are already demonized when you boot up.

The second use is not available to unikernel applications. The limitation of a single application per vm applies.

murderfs• 8 years ago

Claiming that they've "implemented" 42 syscalls seems like an overstatement: almost everything is a stub that does nothing, and the bits that are implemented are not even close to being half-baked. Even relatively easy stuff like dup is missing.

hugelgupf3• 8 years ago

The number of system calls says close to nothing of the actual API surface present here. It's a poor measure. What actually matters is: what can I run?

That said, 42 is not enough to run a lot of applications.

It'll be interesting to get to the point where they can run a language's runtime or unit tests.

Take it from someone who worked on gVisor: this is gonna take a long time, and eventually you'll end up with a system that's just Linux. You'll end up either being bug-for-bug compatible, which means bringing in loads of compatibility code and architecting your kernel just like Linux, or you'll end up having to change applications (and then you're not all that compatible anymore).

Neither are fun or sustainable compatibility stories. You'd probably actually get Linux apps to work on a random OS faster & nicer by porting gVisor.

Moreover, since implementing a system call is typically pretty simple [...]

You're in for a fun ride, folks.

hugelgupf3• 8 years ago

Given what I just read about IncludeOS (I'm not really familiar), it feels like implementing as much of musl as possible is probably the best you can do compatibility wise. There are going to be a lot of Linux features simply incompatible with includeOS, so claiming that 400 syscalls will be implemented is unlikely.

That said - why do you have to get musl compatibility by implementing the Linux syscalls? Is contributing to musl an option?

perbu8• 8 years ago

(I work for Includeos)

I don't think we need to run everything. I think we'll be happy if we can take some common libraries and compile them and they work. That is currently our goal. Down the road, depending on interest, we can perhaps expand a bit.

> Moreover, since implementing a system call is typically pretty simple [...]
You're in for a fun ride, folks.

Thanks for the heads up. It was meant as opposed to implementing the syscall and layers on top of it.

DSingularity• 8 years ago

Do you recommend gvisor over dune or kata containers?

hardwaresofton2• 8 years ago

If you're into unikernels, rumpkernels[0] are the least-modification-required approach I've seen.

Basically, if you compile your program with a minimally (as minimal as is possible) modified compiler, you get "rumprunnable" output. there's a list of popular packages built to run on rump kernels[1] (Rust is supported[2]). I haven't tried to use the toolchain yet, but it's pretty appealing to me.

The biggest issue is that rumpkernels seem to not be in developed anymore. I can't tell if the entire is just off somewhere being productive with their rumpkernels and are happy or if no one is around at all. The Github organization[3]'s repos all seem to not have been changed in a while either.

Outside of rumpkernels, Nabla containers[4] also look interesting to me. They're built on Solo5[5], but the only microkernels on there are currently MirageOS (for OCaml) and IncludeOS (C++?). Nabla looks amazing (containers look like a great fit and they seemed to have improved ergonomics massively), but the limitations[6] are intense.

[0] https://github.com/rumpkernel/wiki/wiki/Info:-FAQ

[1] https://github.com/rumpkernel/rumprun-packages

[2] https://github.com/rumpkernel/rumprun-packages/tree/master/r...

[3] https://github.com/rumpkernel

[4] https://nabla-containers.github.io/

[5] https://github.com/solo5/solo5

[6] https://github.com/nabla-containers/runnc#limitations

perbu8• 8 years ago

I'm hoping we're be able to port libuv to IncludeOS (which should be doable, much like libuv IncludeOS is eventdriven). Then we should be able to get Node up and running. Running Node.js through IncludeOS on Nabla could be interesting in a "FaaS" context.

Last I heard the author of Rumprun has picked up beer brewing full time and isn't interested in maintaining the project.

It would be great if someone where to pick it up.

OSv is also very relevant. They provide a much richer and more compatible runtime environment compared to IncludeOS (I think they even have zfs). And OSv is pretty active these days, so if you looking for a way to run some high performance stuff OSv is worth looking into.

hardwaresofton2• 8 years ago

Thanks for shedding some light on the possible status of the Rumprun maintainer! Hope he's happy brewing beers :)

Thanks for mentioning OSv[0], I'd never heard about it before.

I'm excited to see what happens with Nabla, I read the accompanying paper and was pretty excited about the possibilities. Nabla's greatest strength is it's ergonomics, IMO, though rumprun comes close (someone took some time to make dockerized containers for it).

[EDIT] - looking at OSv's (primary?) tool capstan[1], I'm not sure it's quite the same, it seems more like linuxkit[2] than rumprun. Maybe my intuition is wrong, but it seems much more like you're bringing along a stripped down OS than you're writing a program meant to interact with a shared unikernel. The distinction is so fuzzy in my head I'm not sure there is one, but it feels different.

[0] http://osv.io/

[1] https://github.com/cloudius-systems/capstan

[2] https://github.com/linuxkit/linuxkit

zapita• 8 years ago

There’s a lot to like here.

1. Focusing on compatibility is smart because it’s the weak point of IncludeOS and other unikernel projects.

2. Musl is an incredibly underrated project and I hope one day it becomes the new standard libc.

This is just a first stab, but it could turn into something great.

I wonder if gvisor and includeos could be combined in interesting ways, or at least steal from each other’s code and ideas?

4ad• 8 years ago

So, what's wrong with newlib?

ZiiS• 8 years ago

Nothing; it is just much smaller by not providing POSIX and Linux like interfaces.

Alpine proves full musl can already run a full distro's worth of programs and is very easy to port anything using glibc.

Newlib tends to be used in projects like Cygwin as a small C standard library, where the rest of the POSIX API is implemented separately (by proxying to Win32 system calls for Cygwin).

jetzzz• 8 years ago

Looks like Linux syscalls are becoming some kind of de-facto standard. They are already implemented by Linux, FreeBSD, Windows and now IncludeOS.

AnIdiotOnTheNet2• 8 years ago

The mind reels.

Not because Linux is a bad kernel, mind you, but because UNIX itself reeks of ideas that were ok when it was originally developed and clunky at best today. Even Microsoft seems to have given up. The future of computing looks pretty grim to me.

solarkraft• 8 years ago

Could you elaborate?

ncmncm• 8 years ago

This is an exciting development that I am watching closely.

It should be noted that whatever the limits of the 42 system calls, network operations are very well supported. Many useful programs hardly need anything else.

leni536• 8 years ago

Would it be possible to run Linux as a unikernel and port your application as a kernel module? How a leaned down Linux kernel would compare to includeOS?

perbu8• 8 years ago

There is a project that tries to turn Linux into a library which could lead to a unikernel approach with the Linux. If you look at https://github.com/lkl you can see where they are.

A leaned down Linux-kernel with the application linked into kernel space would be pretty comparable. But it would be pretty enormous beast.

zaarn• 8 years ago

You can compile the Linux kernel with only the modules necessary for running it in a virtualized environment (ie, only virtio modules) and simply include your application in the initramfs as init. If you include nothing else, that's about 6MB + your application.

emilfihlman• 8 years ago

The article is "broken", it talks about Musl and then says the found Musl.

chris_wot2• 8 years ago

Can someone point me to a more detailed overview of IncludeOS than the brief summary they have on their home page?

perbu8• 8 years ago

For a technical, in-depth overview of IncludeOS and how it was made I find the main authors presentation at Cppcon 2017 is a pretty good introduction:

https://www.youtube.com/watch?v=h7D88U-5pKc (one hour, quite entertaining).

There is also the original paper which was presented at the 2015 IEEE 7th International Conference on Cloud Computing Technology and Science (CloudCom).

https://ieeexplore.ieee.org/abstract/document/7396164

Linux compatibility in IncludeOS

Related Stories