MirageOS: A Unikernel For The Xen Hypervisor

http://openmirage.org/

by godisdad • 12 years ago

99 52 12 years ago

openmirage.org

derefr3• 12 years ago

See also: Erlang on Xen (http://erlangonxen.org/).

alco4• 12 years ago

Also http://corp.galois.com/halvm.

Those three have in common the trait that they support single language runtime: Erlang for Erlang on Xen, OCaml for MirageOS, and Haskell for HaLVM.

derefr3• 12 years ago

Which seems sensible; if you want to write different components in multiple languages, each component can be its own VM and they can communicate over the (virtualized) network. It's the natively-compiled, statically-linked equivalent to putting your runtimes in Docker containers.

I was thinking, earlier, that this approach could be taken even further: rather than relying on Xen, it'd be nice to extract the drivers from the Linux kernel into their own project, an exokernel library with a defined ABI. Platform runtimes like Erlang, OCaml, Haskell, etc. could each build a "complete bare-metal kernel-binary" version of themselves, simply by linking to that exokernel library.

(And Linux could obviously move over to consuming the exokernel library itself. I guess a POSIX environment would technically just be the "complete bare-metal C kernel-binary.")

ams6110• 12 years ago

Also https://github.com/kerlnel but it doesn't appear to be active.

andrewstuart6• 12 years ago

This is an awesome idea. Every now and then a revolutionary concept comes along that points the way to all future computing. This is the future of any public facing computer. You must understand the concepts here if you are involved with web development at any level. If you haven't yet wrapped your head around this, then you should.

The key concept here is about security.

Imagine you could compile your web application/website into an application that has nothing else but the required network drivers and minimal hardware drivers needed to execute. That is (as far as I can tell from a brief look) what Mirage does. Using such an approach there's nothing for hackers to log in to - no shell to use, no other insecure cruft on the system potentially leaving holes. The attack surface of such a system is extremely low relative to putting fully powered and configurable interactive operating systems onto the Internet.

Hackers are showing that almost everything is hackable without expensive, constant, expert attention to security of every aspect of your systems. Even then zero day exploits expose systems within seconds or minutes of the exploit becoming known.

This is not to say that such systems cannot be hacked, but rather to say that in such an approach the attack surface is dramatically lower than in current approaches to deployment.

Perhaps more likely to change the way that people think about the concept of web application development in the longer term is that the application/server allows for the possibility of "high resolution server instances" in which a virtual machine can be started in milliseconds, perhaps to service only one query, and then vanish again. Instead of renting your instance from Amazon that runs per hour and does nothing most of the time, your infrastructure will be truly scalable with nothing running at all until its needed and then precisely as much computing power will be used as is required to meet the inbound demand. No longer will you need to "start a web server" which then chugs and rumbles into existence as all its various unneeded-at-most-times processes start. Instead, connect your application/server AMI to the Amazon spot instance marketplace.

alco4• 12 years ago

That is partly true. But if there is nothing to hack into, there will also be nothing to log in to for an admin or an ops person.

In the case of Erlang, for instance, it provides a built-in remote shell which allows one to connect to a running Erlang process, get multitude of information about the VM, loaded code, etc.

As long as there is a convenient way to maintain a running system, that same way can be used for hacking.

andrewstuart6• 12 years ago

Perhaps deployment and management practices will change to take into account this approach to management, rather than forcing the existing insecure approach onto dedicated applications/servers.

amirmc5• 12 years ago

You don't always require a shell if all you need is info about the system. Such a logging process can easily be compiled into the application and that data regularly pushed elsewhere. If you need to change what's being logged then adjust the code and recompile/deploy a new Unikernel. I have a version of my website being compiled as a Unikernel and the workflow is very simple [1].

My point is that the whole workflow of creating and deploying applications becomes much simpler as the final VM can be an (almost) disposable item, rebuilt at-will. This is pretty much what the configuration management tools are all striving for but Unikernels work this way by design. Therefore, our notions of managing such systems will also change.

[1] http://amirchaudhry.com/from-jekyll-to-unikernel-in-fifty-li...

anonymousDan• 12 years ago

The main issue I see is that everything has to be done in ocaml no? What about the maintenance overhead too?

andrewstuart6• 12 years ago

The dedicated application/server is such a powerful idea and security is such a big problem that perhaps some time in the future people will see the idea of "directly putting an operating system on the Internet" as risky madness. All development environments that matter will somehow come to support creation of dedicated application/servers.

rjsw• 12 years ago

There is nothing stopping someone from doing something similar in a different language.

I made a start on modifying Movitz to build a Lisp system that would run on top of Xen.

andrewstuart22• 12 years ago

Holy crap, it's you! Sorry I tried to reset your password the other day. I thought I'd already signed up and forgotten.

On a more on-topic note however, aside from the benefits of running any kernel, such as windows, I wonder if there are any practical benefits of something like this vs something like CoreOS/docker. I'm sure since it's Xen you could run a windows kernel on Linux, but aside from that, I don't know.

andrewstuart6• 12 years ago

Holy crap, it's me! I did get the password reset in the middle of the night and wonder what that was about then I saw andrewstuart2 and realised what had happened. Seems we are the same person. Do you spend your life saying "It's spelled S-T-U-A-R-T", then people say "errrr, S-T-E-U-W-A-R-T?".

avsm3• 12 years ago

On a more on-topic note however, aside from the benefits of running any kernel, such as windows, I wonder if there are any practical benefits of something like this vs something like CoreOS/docker.

There are millions of lines less of code involved in the deployed Xen unikernel, since there's no Linux userspace/kernel divide involved any more.

You can also use the same application source code just fine with CoreOS/Docker if you prefer, since it can also compile to normal POSIX binaries that use kernel sockets (via `mirage configure --unix` instead of `mirage configure --xen`). This is the main benefit of using modular OCaml code that can swap out entire subsystems at compile time.

harrystone• 12 years ago

This seems like the rube goldberg version of chroot to me.

p1mrx• 12 years ago

There was already a "MirageOS", for TI calculators:

http://www.ticalc.org/archives/files/fileinfo/139/13949.html

Sir_Cmpwn• 12 years ago

One with a frustrating name. It's a shell, not an OS. One of my projects is an OS for TI calculators, and loads of people don't actually understand what that means.

gtirloni2• 12 years ago

I understand how using type-safety and streamlined modular systems might be good for security, but I fail to see the point in letting developers be closer to the bare metal. All the security abstractions built into traditional OSes are not there for no reason. Why should I trust the OCaml stack more than the alternatives?

This looks like something a really educated/disciplined company could use for creating some crazy cool infrastructure. It's hard to see it taking off for the average Joe.

amirmc5• 12 years ago

I think you answered your own question (to some extent). Rewriting the necessary protocols/libraries in a type-safe manner already provides an improvement over the current systems. In addition, (with the standard approach) it's quite hard for a developer to reason about the whole stack they're using and privilege escalation bugs are particularly irksome. Writing the applications with the approach Mirage takes means that there is less code to reason about.

An anecdote I use when describing the benefits is the story of a smart fridge that got hacked and became part of a botnet sending spam emails. Why did that fridge even have code that allowed it to send email? It wasn't necessary for its functioning. We should write software differently if were going to be deploying it to 10x the number of devices compared to today.

More personally, I worry about the software that's going to find it's way into the embedded health devices of the future (cf pacemakers). These devices will inevitably be 'connected' and I want to make sure that the code they use is safe and secure.

brokenparser• 12 years ago

Why did that fridge even have code that allowed it to send email?

If it has a remote code execution vulnerability, it's trivial to make it send spam (or do all kinds of things) whether a MUA was already present or not.

godisdad• 12 years ago

OP here. The Unikernel paper someone linked about has more, but I think the main ideas are: incidental complexity mitigation and performance.

It's much easier to reason about a bunch of OCaml code, so say the authors, than it is to understand the interaction between independently changing pieces of your "stack" that are written in different languages and integrate in disparate ways.

bjackman• 12 years ago

I don't work in a sphere that would make this relevant to me, but I think this paradigm is probably the future! "General purpose operating systems are dead!", proclaimed bjackman.

Anyway, also see: OSv: http://osv.io/ similar idea but with the JVM. I believe some pretty well known Clever People are working on it - can't remember who, though.. anyone know?

gtirloni2• 12 years ago

What looks like an alternative in .NET: http://cosmos.codeplex.com/

listic• 12 years ago

Is there anything similar for KVM?

justincormack3• 12 years ago

I don't think so. KVM provides something much more like raw hardware (provided by qemu) while Xen provides a slightly higher level interface, and an example "OS", mini-os to run on it. Your best bet would be to use a virtio based driver set and a minimal initialisation but it will still have to do more.

edwintorok8• 12 years ago

According to the FAQ it should be possible to port it to KVM: http://openmirage.org/wiki/faq

amirmc5• 12 years ago

For people interested in this I recommend the ACM article on Unikernels [1]. It's a good overview of the approach and the benefits/trade-offs. Then take a look at http://nymote.org to see the kind of things we want to start building with this approach.

[1] http://queue.acm.org/detail.cfm?id=2566628

ForHackernews• 12 years ago

There's only room for one MirageOS in my heart, and it runs on the TI-83+ http://www.detachedsolutions.com/mirageos/

bsaul• 12 years ago

Tried to read the paper mentionned on the website, and wondered : does "libOS" /unikernel concept means that the application is built as a kernel extension, and thus runs in kernel land ( so no context switching occures) ? Is this an explanation for the performance gains ?

edwintorok8• 12 years ago

When you link the application with the Mirage runtime (what you refer to as "libOS") you create a new kernel. You then run that kernel directly on Xen (like you would run a Linux domU), so yes they run in kernel land.

I'm not convinced that 'Xen + Mirage unikernel' has a performance advantage over 'Linux baremetal + Mirage direct mode', but I definetely see an advantage compared to 'Xen + Linux domU + Mirage direct mode'.

MirageOS: A Unikernel For The Xen Hypervisor

Related Stories