Rethinking the guest operating system

http://lwn.net/SubscriberLink/567222/d02ddd1c4e25ae22/

by justincormack • 13 years ago

120 48 13 years ago

taproot2• 13 years ago

This sounds like a great opportunity for anyone wanting to get into os development with a few easy tasks still todo.

I'm curious how you would configure and manage your applications. Like are you able to attach to the input and output streams from the host or would you still get some basic form of bash to manage it?

rwmj3• 13 years ago

A lot like: http://www.openmirage.org/

This is an operating system that runs directly on the Xen hypervisor, written in OCaml, which does away with the usual OS abstractions.

masklinn• 13 years ago

Or http://erlangonxen.org/ which does away with a "traditional" OS entirely, running an erlang runtime directly on top of Xen.

rwmj3• 13 years ago

Right, they all sound similar.

What the article doesn't really emphasize enough is the aim of Cloudius is entirely about running the JVM. This of course will guarantee popularity amongst "enterprise" types.

However it's still very similar to running a JVM in a process directly on the host. You could do something similar by running a JVM on the host and using cgroups to confine it.

Cloudius's USP is that existing clouds are already running full guest operating systems, so their OS-v running a JVM fits into this landscape naturally. Architecturally it's nothing new.

I wish Dor & Avi well though :-) (ex Red Hat associates)

nly• 13 years ago

While interesting for bespoke applications the problem is, if you take this concept to the extreme, you end up with a traditional operating system... except it'll only run Erlang.

mercurial• 13 years ago

Also https://github.com/GaloisInc/HaLVM (Haskell). Though I think it doesn't handle parallelism.

pjbringer• 13 years ago

Stanford has another somewhat related project: http://dune.scs.stanford.edu

In their case, the application runs both as a process in the host and as a guest. It gives the application access to traditional OS APIs, and allows use of processor extensions to directly access virtualized hardware. The benefits of doing this include ability for an application to do low level custom IPC, to use the page tables for garbage collection, to trace the use of system calls much more efficiently, to hook into page faults, and so on. Very cool stuff.

ogrisel3• 13 years ago

Will this make it possible to use docker.io directly under OSX without having to use vagrant / virtualbox to run a linux host OS for LXC containers?

Right now, only Xen, KVM and EC2 HVM are supported hypervisors. Hopefully an OSX might come with vmware support later.

lotyrin3• 13 years ago

No, this is sort of the opposite of containers.

Instead of having an full OS-like isolated system within a single OS without virtualization, this is using virtualization but avoids having a full OS-like system inside the guests.

ogrisel3• 13 years ago

Yes I understood that. But right now if you want to use docker.io on your OSX dev box you have to run linux inside a VM to be able to leverage the docker.io features (building and testing apps in lightweight containers) for later deployment on the target cloud infrastructure.

If you have OSv working both under virtualbox or vmware on your OSX dev box and under KVM or EC2 HVM on your cloud production environment, then it might be possible to have a docker.io features directly under your OSX dev box.

hrjet2• 13 years ago

I am not sure why this is opposite of containers?

Wouldn't it be possible to run a minimal POSIX OS inside a container? (genuine question, I don't know the answer to)

nl2• 13 years ago

Wow.

I've played with CoreOS a bit, but this is a much more radical change.

I love how people are beginning to rethink many of the things that all successful operating systems have had in common so far.

The idea of using virtualization as an inherent layer in the application architecture (ht IBM OS/360) is great for flexibility.

kalleboo• 13 years ago

So in the end, we're reinventing the OS with a hypervisor acting as the kernel? Are the current hypervisors better or worse than the Linux kernel at this task? What's the reason people don't just run these processes straight in a regular Linux install with properly configured users/chroots/quotas?

harrytuttle3• 13 years ago

Yes, processes are a long forgotten form of virtualization. You are spot on there.

They are pretty good :)

ddalex• 13 years ago

cgroups help solve the same problem, for tasks.

I suspect running properly isolated JVMs directly on host kernel would be faster than going through the hypervisor. I would love to see benchmarking numbers.

But there are two problems on why people keep reinventing the wheel:

- Current solution Isn't Sexy Enough, aka Not Invented Here.

- Current corporate culture drives acquisitions of tech startups based on a) business, i.e. number of users, and b) innovative technology that doesn't exist anywhere else, reinforcing previous point.

All this work to come up to yet another solution to the same old problems (IBM solved machine partitioning around 70s ?) is happening because engineers love to work on their specific pet projects and not solve other people's problems.

Not that there's anything wrong with that.

jadc• 13 years ago

Just off the top of my heads, some benefits of VMs vs processes: - you get the encapsulation/isolation out of the box - abstraction from the hardware; may not seem like a big deal if you're thinking in terms of a single machine but very useful for migration (whether live migration or cold migration) between different machines - ability to snapshot the state - arguably a smaller attack surface since you deal with fewer drivers inside the virtual machine versus a general purpose OS (VM escape exploits do exist though - no denying that).

rjsw• 13 years ago

Using a VM can also reduce copying of data, particularly in the network stack.

Negitivefrags• 13 years ago

I for one, really dislike this drive towards VMs for everything.

VMs are mostly about solving the same problems that operating systems were there to solve in the first place, only slower.

I personally think that part of the problem is the way software is installed and configured by modern package managers and distributions. It makes people see the installed software as part of the operating system almost. If you want two webservers with different configurations you therefore need two operating systems.

The concept for creating a couple of different users and running the software out of each one seems foreign to modern system admin these days.

arethuza• 13 years ago

One additional reason for the popularity of VMs in enterprise environments is that software vendors will often refuse to support applications on servers that have other applications installed.

Supporting high-availability of guest servers by allowing "live" migration from one host to another is pretty cool as well - either for fail-over or to allow hardware maintenance.

The default position these days in enterprise environments seems to be that a server will be a VM unless you have a really strong case for having dedicated hardware (and that only ever seem to apply for database clusters).

roarroar• 13 years ago

The sole reason for Xen etc is that people were too lazy to do microkernels in the first place. It is exactly the same thing as we are seeing in the web browser with all the low level primitives like asm.js and WebGL: the good ideas inevitably start to creep back in (even if in a crap, watered-down form at first) given enough time. In fact, we could easily have proper MMU sandboxing in the browser by now if we were running proper microkernels. Instead we'll pay big performance and complexity overheads to do it in userspace.

vidarh• 13 years ago

Just creating a couple of different users does not give nearly the level of isolation a VM does.

rwmj3• 13 years ago

Architecturally, there's no real difference.

However in the real world there are self-service clouds which offer cheap, easy to consume VM containers. So having a JVM which runs directly in these containers makes some sense. (The alternative would be to persuade Amazon to let you run your JVM as a host process using LXC or something .. good luck with that)

lotyrin3• 13 years ago

I guess I'd have to see some compelling benchmarks of a JVM on OSv vs a JVM in a more traditional VM image.

mwcampbell4• 13 years ago

OK, so run your JVM on something like Heroku or dotCloud that does use LXC. Too bad those are in turn running on EC2, not on bare metal.

justincormack• 13 years ago

The hardware has added a new form of isolation in addition to just memory protection though, that the process alone can't use.

mwcampbell4• 13 years ago

This isolation comes at the cost of dramatically decreased performance.

http://dtrace.org/blogs/brendan/2013/01/11/virtualization-pe...

Hardware virtualization is best viewed as a hack for running multiple operating system kernels at the same time, where each kernel is designed to have a machine to itself. In any sanely designed system, this shouldn't be necessary; multiple processes under a single shared kernel should be good enough.

pantalaimon• 13 years ago

This seems a bit ridiculous to me.

In the beginning you had

[OS] -> App

Then, people would put those Apps into a VM, the trend going to one VM per app.

[OS] -> [VM] -> [App]

Just to realize that the VM may be too much of an overhead, so now OSv comes along to cut that down, relying on the OS for memory management, task scheduling, etc, effectively ending up with

[OS] -> [translation layer] -> App

So that's just a glorified sandbox, why not just use LXC?

harrytuttle3• 13 years ago

Actually just screw all of that and have:

[OS] -> App

Cheaper, less administrative overhead, less abstraction, less vendor tie in (if you go POSIX for example).

I think that might upset the virtualization proponents though...

radq• 13 years ago

Wouldn't that be less secure? I understand that it's possible to use chroot and stuff, but isn't the whole point of virtualization/LXC that you have better isolation and more control over quotas and stuff?

I am just curious as to why people would use virtualization at all if it is possible to accomplish the same thing using regular processes.

a-priori• 13 years ago

This is my ideal solution as well. Processes are already fully isolated just like virtual machines or containers. They may only execute during time slices where the kernel schedules them, can only communicate and access system resources through the system calls the kernel provides them access to.

If processes are insufficiently isolated, it's the system call interface that's broken, not the isolation model.

It seems to me that virtual machines and containers could be implemented on top of the existing process hierarchy by allowing a parent process to intercept and reinterpret its child process' system calls. Simple example: Want to implement chroot? Intercept all open() calls and prepend the root path (taking care to prevent escaping with '../').

pjmlp• 13 years ago

With OS/360 actually you already had

[VM] -> [OS] -> [App]

Which is the approach used by Windows Hyper V and VMware vSphere Hypervisor.

qznc• 13 years ago

For example, you can improve performance by using copy-on-write via the MMU. However, Linux/Win/OSX will not allow you access the MMU. Within a VM you can do that.

pibi• 13 years ago

anttiok• 13 years ago

Someone asked me to compare the two:

http://blog.netbsd.org/tnf/entry/running_applications_on_the...

tl;dr amount of code rewritten vs. reused

jbellis• 13 years ago

1000 words on why this matters: http://www.slideshare.net/eonnen/high-performance-network-pr...

mwcampbell4• 13 years ago

If the virtualization tax is high when you care about throughput, then isn't that a case for avoiding virtualization altogether?

mwcampbell4• 13 years ago

Like others here, I am unconvinced that there is any benefit to basically using virtual machines as heavyweight processes. The existence of at least one multi-tenant IaaS provider (Joyent) and a few multi-tenant PaaS providers (Heroku, dotCloud) using OS-level virtualization suggests that a shared kernel running multiple processes provides enough isolation.

yalogin• 13 years ago

I am not so sure. I would say there is a market for both. There are things that LXC based solutions cannot do yet. Solutions around VMs are very mature and offer features like live migration of apps for e.g. by which I mean they even do monitoring of apps running on the data center.

justanother• 13 years ago

So we have a guest operating system that doesn't implement multitasking/timesharing because you can safely leave that up to the hypervisor. Is it just me, or is this more or less the same thing that was done with VM/CMS on IBM 370s in 1972?

Rethinking the guest operating system

Related Stories