What are Linux containers and how did they come about?

http://bitmason.blogspot.ca/2013/09/what-are-containers-anyway.html

153 63 13 years ago

bitmason.blogspot.ca

derefr2• 13 years ago

In practice, containers are process groups with access restrictions.

In theory, though, containers are simply an optimization of virtual machines, and are best understood that way. If you can use virtual machines to solve a problem--and your virtual machines are all generally based on a recent Linux--then you can achieve the same thing, using far fewer resources, by using containers.

3am• 13 years ago

I think the virtual machine analogy is a bit dicey because lxc+cgroups/Docker containers have to be much more aware of the host kernel api and devices.

edit: yes derefr, I agree. My point was also if there was a distribution or kernel specific Ruby or Ruby library bug, for example, then you might have to account for it in your code if you ran in containers (if you don't control the host on which your code is running), whereas you control the whole stack in the virtual machine.

derefr2• 13 years ago

But it's not idiomatic to take advantage of those close ties (by, say, having two containers share a named pipe mounted from the host.)

I suppose what you mean is that containers aren't applicable in every instance that VMs are--like, say, providing VPS instances, where some of the instances might want to do funny fiddling things with their virtualized "hardware." This is true.

But for the situations where the applicability of VMs and containers intersect--using them to run Heroku-like isolated "app slugs", for example--then you can think of containers as lighter-weight VMs, and you won't really go wrong.

MrBuddyCasino2• 13 years ago

I think we've used something similar some time ago (maybe XEN?) and had the following issues, how does Docker compare: - what happens when one container changes the system clock? - are iptables rules per container or global?

ambiate3• 13 years ago

As far as I understand, the clock and such are tied to the base host. Typing 'uptime' in my linux container shows 5 days, the uptime of the base host.

djchen• 13 years ago

I was thinking more like FreeBSD jails. Containers can't change the system clock.

qznc• 13 years ago

Networking for Docker containers is bridged, so iptables are per container.

http://blog.docker.io/2013/08/containers-docker-how-secure-a...

mcpherrinm• 13 years ago

Linux network namespaces let you have multiple independent network stacks in linux. That means routing tables, interfaces, routing tables, IP addresses, etc.

Each container gets its own network namespace (along with other namespaces like hostname, pids, users, ipc, filesystem mounts). Anything not handled by one of the 6 namespaces is the same across all containers. That includes things like what kernel modules are loaded, the system clock, etc.

Because a user with root can manipulate the kernel in many ways, I wouldn't give root to an untrusted user and assume containers were enough to contain them. Certainly if they can load a custom kernel module it's game over, but I'd bet there's plenty of other ways to break out too.

yrro• 13 years ago

See the excellent series of LWN articles, "Namespaces in operation" <https://lwn.net/Articles/531114/> for an overview of how namespaces work on Linux.

To answer your time question: AFAIK there is no namespace for system time in Linux. If you don't want processes within a contaier to be able to set the system clock then don't launch them with the CAP_SYS_TIME capability.

k3oni2• 13 years ago

Welcome back OpenVZ... except that this will be part of the OS now.. and i wouldn't call these "an optimization of virtual machines".. containers are part of the kernel/OS so call it kernel/OS virtualization based if you want where virtual machines are or supposed to be hardware virtualization based.

Ex. openvz,lxc - containers kvm,xen - virtual machines

memracom3• 13 years ago

OpenVz was something that required hacked kernels.

A better analogy is LXC = Solaris Zones

KVM and XEN are hypervisors for virtualization. VirtualBox is a non-hypervisor virtual machine.

diminoten2• 13 years ago

Except when you need snapshotting, live-migration, or have a highly disparate set of underlying OS requirements.

The author of the article explicitly says, "This isn't to say that they're going to replace virtual machines." and I agree - there's no point in pretending like they're solving all of the same problems. I guess you could say that containers are an optimization of some use cases of VMs.

gaius5• 13 years ago

But that's simply not true. 20 years ago you could live migrate processes around a VAXcluster with nary a VM in sight.

kev009• 13 years ago

OS level virt can easily do the first two. The third is harder due to a common kernel. But FreeBSD for instance can run CentOS 6 with the Linuxulator. I believe Solaris has similar feature called "branded zones".

justincormack4• 13 years ago

Live migration from a container is pretty close to working (ie works if you are careful). Linux live migration is not whole-kernel it is per resource so you can migrate just a containers resource.

I still do not understand the use case of live migration though...

cbsmith4• 13 years ago

Actually, I've been doing live-migration & snapshotting with OpenVZ for years.

Really, the only reason to use virtual machines is if you truly need different OS kernels.

zobzu• 13 years ago

Thats kinda true indeed. Containers people want to use are VMs that directly talks to the local kernel (i.e. it does not run another kernel on top, like virtualization does).

You can make process groups without making the container look like a VM ... BUT. That's hard and time consuming for little gain.

pjmlp• 13 years ago

That is why micro-hypervisors where all OS instances are virtualized even the one acting as master, like on mainframes, are gaining acceptance.

cbsmith4• 13 years ago

As someone who has used OpenVZ for a long time, I'd have to disagree with the notion that containers are simply process groups with access restrictions. You have a distinct namespace for them (including networking), which really changes everything.

lsc3• 13 years ago

Generally speaking, containers have worse resource separation. Last time I looked, a system with containers shared pagecache.

Sharing pagecache means that one user thrashing the disk can make the system unusable for everyone, well, to the extent that no system without pagecache is usable.

Xen and KVM mitigate this by giving you your own ram and thus your own pagecache.

Personally? I think this makes a way bigger difference than any cgroup limiting of iops. Pagecache is not free ram, and should not be treated as such.

I think containers are one of those tradeoffs; they are more efficient when it comes to hardware, but they are less efficient when it comes to sysadmin time tracking down users who are using more resources than they ought.

On the other hand, some find managing multiple containers easier than managing multiple complete virtual servers.

jsmthrowaway• 13 years ago

Part of the problem, Luke, is that you're thinking about containers in terms of a hosting provider. That's not your fault. You're a hosting provider. To you, cotenancy is not only a security concern but also performance and SLA. For almost everyone else, efficiency is way paramount.

I still think right now the best way forward is:

- Hosting provider: Xen/etc

- Internal clusters: containers

Containers merely bring commodity computing closer to HPC operational techniques. Containers are not useful for you at prgmr. Using every drop of RAM on the machine for page cache is a positive for my use case, not a negative.

Virt overhead makes no sense in internal clusters, in my opinion. I've worked in an all virt environment and an all container environment. The latter is way better, and these are scales where Puppet burning a core for several seconds across the fleet has a demonstrable impact on Opex.

gaius5• 13 years ago

the concept--and products based on that concept--has been around for almost a decade

Closer to 6 decades, this is old hat for IBM mainframes.

For me the question is tho', why virtualize? And the answer I keep coming back to is that a) it is too hard to retrofit good manageability onto processes as the basic unit of applications and b) people are used to it, the idea of having 1 machine per app, when really it isn't cost effective to do that.

ambiate3• 13 years ago

Segregate applications. If app one gets compromised, app two is off in another land. If my linode box gets hacked, without containers, all 16 websites will be compromised/go down.

Desktop virtualization using the NX protocol and Linux containers is also really friendly. Our company uses old Pentium D boxes for desktops. The performance gain from using a thin client + lxc + lubuntu is insane! I can actually watch youtube videos and draw in OpenOffice Writer. (Also great for disaster recovery scenarios).

gaius5• 13 years ago

But processes - theoretically - are segregated. Own address space, running as different users, bound to a processor set, etc. The use of hypervisors is because processes in practice don't really do what they say on the tin, so you need to force another layer of protection and manageability in, and just eat the overhead.

slig• 13 years ago

This is the current text:

to change the text of this page send 0.1 btc to 16JMNc3B5vCkuuPxNcNj388gmhP8UDKBuW

Well done, it didn't take more than 1 hour to someone find a loophole.

gaius5• 13 years ago

Isn't that $100? The mind boggles.

ghayes• 13 years ago

Wrong thread?

gtrubetskoy• 13 years ago

This article curiously fails to mention Linux VServer http://linux-vserver.org/ which has done this since 2003 and unlike Virtuozzo has always been completely open source. We've used it with great success in a hosting product.

ghaff3• 13 years ago

I probably should have mentioned it. Virtuozzo was much higher profile at the time I was closely following various workload separation technologies--in part because it was making a serious attempt to position itself as an alternative to VMware in the enterprise space. (Something it ultimately failed to do although it continued to enjoy considerable success in hosting.) The fact is that there have been lots of different isolation approaches, including OS virtualization, and Virtuozzo's implementation of OS virtualization--while hardly unique--was the best known along with Solaris Containers.

cbsmith4• 13 years ago

It also didn't mention OpenVZ by name. Honestly, ever since OpenVZ came out, I haven't felt like the case for using VServer was terribly strong.

octix3• 13 years ago

Can anyone please give a simple example or link of a use case of containers need?

I use KVM/Qemu VMs at home and work(dev box), I have a dozen always running VMs. However, I started to read more and more about LXC containers, and still cannot grasp their importance... for example, I manage a few VPSs too, each has it's own purpose (web server, database), how could LXC help me?

Thank you in advance.

cbsmith4• 13 years ago

Basically you can use them just like you use your VM's. The main difference is you lose the ability to have different kernel's for each VM.

octix3• 13 years ago

Ok, thank you.

ghaff3• 13 years ago

Basically every Platform-as-a-Service uses containers. They're much more efficient than VMs (resources, spin-up/down times) and the underlying infrastructure is standardized anyway. They may not very well not be relevant to a couple of home systems. We're talking about large-scale infrastructures.

octix3• 13 years ago

Yes, fast spin-up/down are an advantage.

Thanks.

What are Linux containers and how did they come about?

Related Stories