Why is Docker the new craze in virtualization and cloud computing?

https://opensource.com/business/14/7/why-docker-new-craze-virtualization-and-cloud-computing

by jhibbets • 12 years ago

78 61 12 years ago

jhibbets• 12 years ago

My favorite quote from the article/interview: "Its mission is basically: make it easy to package and ship code"

j45• 12 years ago

Docker to me makes things an order of magnitude (or two) easier than previously possible. In time these additional layers will be as central as virtualization itself was in arriving with VMware.

emeraldd• 12 years ago

I'm just start to play with Docker, it seems like a nice way to build out contained systems that you don't want to change much ... but feels like it could lead to problems when it comes time to do updates to software inside the container. It feels like Docker offloads alot of responsibility for security updates from the ops to people in development. Or am I missing something ...

chc• 12 years ago

When you want to do updates, you can just replace the old container with a new one reflecting the updates.

Leynos• 12 years ago

The conceptual leap that I missed initially is the use of volumes, either mapped to data containers or to directories on the host system. These allow the persistent data used by the application in the container to live on, and for the container itself to be seen as immutable and easily replaceable.

vidarh2• 12 years ago

It forces co-operation. As someone doing ops, and with years and years of development: ops teams should not do software upgrades without cooperating with developers and re-testing the full stack anyway. The odds of seemingly "safe" OS upgrades breaking applications are often high.

If used properly Docker enforces pushing those upgrades back up the pipeline as far as they should.

shykes• 12 years ago

That's exactly right. Upgrades should be higher up in the process and involve the development and testing team. It doesn't mean the same people who run production can't also participate and offer guidance in the upgrade process upstream.

nubs• 12 years ago

Docker provides more flexibility. So while you can have developers managing security updates that is definitely not the only way of handling things.

I see a process where ops are responsible for the base images (base os images, and layers for different software stacks) and can handle security updates at will. Developers can maintain small dockerfiles that just install their application on top of the ops-managed image. ops can then be in charge of building (and re-building) the images and deploying them. Of course, that can all be fine-tuned to your liking.

revscat• 12 years ago

I mean I see a lot of folks doing hyper-scaling with containers, indeed you can get a lot of containers on a host and they are blindingly fast. But that doesn't excite me as much as people using it to automate their dev-test-build pipeline.

This is it exactly. When I can develop locally using my favorite editor and my favorite OS, but then deploy -- still locally, mind -- to what is more or less the production environment... This has huge benefits for avoiding the "works on my box" problems that arise as code makes its way to prod.

mpdehaan2• 12 years ago

Run ALL the distributions and versions in prod? :)

vidarh2• 12 years ago

I'd rather have the opportunity to re-test and re-deploy one component at a time than be forced to do "big bang upgrades".

In the long run I hope we start seeing distros or tools targeted specifically at building minimal Docker images. E.g. if I'm going to deploy a Ruby based web app in Docker, the stuff I need present in the container is _very_ limited.

PaulHoule• 12 years ago

No, if your environment is standardized and stripped down you can run something that is very close to prod in your dev environment.

Note docker isn't the only answer for this, for instance, you can use ordinary VMs with vagrant, but docker does it at ludicrous speed.

Kiro2• 12 years ago

They keep saying Vagrant and Docker are two different things but isn't that the whole point with Vagrant?

NDizzle• 12 years ago

Your username is the same as my (jindo) dog! So I'll reply to you.

I've been using vagrant to fix exactly the problem of "works on my box" for years. Docker is way too much to explain to my small development team. They struggled to deal with vagrant for some time before figuring it out.

Vagrant is perfect for us right now, and it would take a force of nature for me to even begin to push docker on them.

revcbh• 12 years ago

In many cases, Docker can be treated like a more lightweight Vagrant. One important benefit I get is more efficient use of machine resources. Running 5-10 fully virtualized Vagrant instances on my laptop hurts. Running the same number of Docker containers is trivial.

The ability to version your images and easily build on them is nice too.

quaunaut2• 12 years ago

Yes! But using Docker can help several problems:

First time provisioning of a box can take up to 30 minutes even with a non-mature app. Mature apps with many moving parts can easily get near an hour- using Docker, you can bring that down to 5 minutes or less, with the majority of that being downloading the containers.

Neat thing though, is that you can use Docker WITH Vagrant to do this. The setup I'm doing on my side projects, essentially goes like this:

If I'm developing, I use Vagrant to spin up Docker containers(using Docker as the provisioner) for supporting items- databases, memcached, things like elasticsearch or a message queue- then develop from my machine(a mac). If I want to get someone else in on the project, I don't have to care one iota about what they're on, Docker will make it work.

On top of that, there's the capability to have your app itself running in Docker containers for development. This means no having to worry about dependencies, environment variables, etc- and if you keep things up to date, you can even just upload your app to a private Docker repo, and use that to deploy to your server(though in a team this may not be the most realistic option at the moment, I've not given it a shot, though I don't see why it wouldn't work).

For me, Vagrant is the tool I use to tie everything together for development. Docker is what I have it run off of, instead of Chef/Puppet/Salt/Ansible, so that my build times go from 30 minutes to whatever download speed I get.

ForHackernews• 12 years ago

Docker is great because it allows you to avoid simulating an entire virtual machine.

Think how ridiculous it is that in many cases today we run an entire simulated computer, with a full general-purpose operating system, just to power a single application (e.g. web server, database, etc).

crdoconnor• 12 years ago

Why is it so ridiculous? It's pretty cheap these days.

adeptus• 12 years ago

It's not ridiculous at all, given production deployments need segmentation to maximize security. Docket may be fine for Dev environments with no real customer data being used, but to deploy to production if you use Docker, you will get shot... by the security guys.

_dark_matter_• 12 years ago

There are plenty of wasted resources. Virtualization itself incurs a 30% overhead.

wfjackson• 12 years ago

It's nice to have logically independent code bases on separate environments just so that they don't step over each other toes with dependencies etc. and can be moved around easily.

octix5• 12 years ago

I don't understand the part with scaling with containers on the same host. I mean if it's the same host, what kind of scaling is it?

scott_karana• 12 years ago

Yeah, I have some guesses but if anyone here can clear that up, I'd appreciate it.

kilburn2• 12 years ago

The scaling part is generally about running and managing multiple containers on multiple machines. In that sense, the advantage of going from virtualization to containerization is the same as going from bare-metal to virtualization: better computation density and more agile deployments.

On the one hand, containers are more efficient than virtual machines. Hence, you get more efficient usage of your hardware. Also, you can pack more containers in a given host than if you use virtualization. Net result: less hardware to run the same services. In turn, this also readuces the network load because the more services in the same machine, the less traffic on your network and better latencies between colocated services.

On the other hand, containers (as envisioned by docker) are much faster to deploy. You build images which are compositions of layers. You have a base layer containing the base OS, and then a "stack" layer containing your application stack, and finally your app on another layer. An image is a consolidated view of all these layers, but the layers can be fetched and managed independently. Hence, you can deploy containers much faster (no need to re-download every layer each time you want to deploy a new service). Additionally, they build faster and they launch faster (nearly as fast as a non-containerized application). This allows you to do even cooler things such as socket-activated containers, which would be painful with the startup times of virtual machines.

Obviously, this blends very well with the idea of building applications in terms of distributed services (or components or whatever). As a result, there are multiple projects pushing for "cluster-aware" (replicated, varying levels of consistency) services. People from the virtualization camp was already pushing for that, but now it is even more important because containers are leaner and cheaper to deploy.

octix5• 12 years ago

Thanks for trying to explain.

In my mind scale means use more resources, but in this case I fail to see benefits, except quicker env setup for the app, but for that there are automation services.

Yes, cluster-aware apps, if I want to scale 1 app, I'll create dedicated nodes/vm instances for it, how are containers helping? Why woudl I need multiple containers on the same host, for each app instance?

Thanks again.

fideloper2• 12 years ago

Docker is operating system level virtualization. Unlike hypervisor virtualization, where virtual machines run on physical hardware via an intermediation layer ("the hypervisor"), containers instead run user space on top of an operating system's kernel. That makes them very lightweight and very fast.

This is one of the best explanations of Docker I've read. (Altho I suppose it requires knowing what Hypervisor is, if just at a topical level...perhaps the same for a "kernel")

evol2625• 12 years ago

Except that this also describes vserver, lxc, solaris zones, freebsd jails, and numerous other container virtualization solutions.

Docker is best described as application virtualization.

Kiro2• 12 years ago

And we're back to square 1. What does application mean in this context? I have a really hard time figuring out what Docker really is.

bithush• 12 years ago

Because Docker just makes more sense. Why should I run multiple versions of the whole OS for near-to-identical setups? That is just a waste of resources. Docker makes much more sense when you have multiple applications you want in separate instances on the same stack. There benefits far outweigh the disadvantages in my experience.

octix5• 12 years ago

Yes, I can see a great deal of benefits for development part, _if_ you're sticking with the same OS. Currently I have a bunch VMs (using kvm/qemu + libvirt) that are created and dropped as needed. The biggest advantages I see is that I can create a near identical environment with our VPSs... + I can test different OSs too (Linuxe vs *BSD).

My current pain point is the hdd space, since I have to resize VMs storage when needed. Even though it was automated, but still I'd prefer to use some shared hdd space and not to worry about it.

jokoon• 12 years ago

can it help game developers ?

quaunaut2• 12 years ago

Probably not at current. Most game developers, for one, develop for Windows. Even if you are developing for Linux, this would only really help if you were exclusively Linux.

CSDude• 12 years ago

I'm using Docker as different than most people intended: To grade student programming assignments on different physical machines. And I can distribute the docker images to students, so they stop complaining "But it works on my machine". So it is a different usage of packaging and shipping code.

fideloper2• 12 years ago

Sort of - the 'works on my machine' issue is definitely something Docker helps with in DevOps settings too.

tlrobinson3• 12 years ago

IMHO every open source project should come with either a Vagrantfile or Dockerfile, if only to allow potential users to quickly try out the project in a known good environment.

Of course then you risk developers of libraries/tools only developing/testing against that particular environment, but that could be solved by having a variety of Docker/Vagrantfiles integrated with a CI system.

justincormack• 12 years ago

That is very fragile, you want to make sure your code is not that dependent on a very specific environment really.

jaequery• 12 years ago

After having used many years of virtualization stacks (openvz, xen, vmware, kvm), I believe using docker on production is still quite tricky, if not very tricky. Especially for larger apps, at the very moment it's like juggling eggshells. It requires a great deal of orchestration as you gradually involve many docker instances and especially when the host is involved (volumes, links). So yeah , I wouldn't call it actual "containerizing" the app and it's more like containerizing "processes". I for one would love to see a day where openvz gets docker-like cli and having repository for creating/pushing/pulling images.

JoshTriplett• 12 years ago

Containers are the new craze, because they're much lighter-weight and easier to deploy than full-system virtual machines. Docker is simply a convenient approach to container distribution and deployment, designed to package up an application as a self-contained invokable container.

Svenstaro2• 12 years ago

Excuse me, but isn't this essentially chroots reinvented?

VLM2• 12 years ago

That is exactly it, with a couple more isolation features added. So more than just the FS but also some memory / cpu limiting and general sandboxing.

Interestingly most "real virtualization" requires somewhat recent hardware but LXC containers don't take much if any, so "free machine off junk heap" is good enough for experimentation.

Historically there have been strange intercontainer isolation problems so don't assume its as perfect as hardware assisted virtualization.

Also you're sharing a kernel which is both good and bad, if you were hoping to test out a new kernel or run an entirely different OS, thats too bad.

Edited to add, another fun analogy if you like the chroot analogy, is when spinning up a process was glacial on windows (but not too bad on linux) that lead to the development and push to use threads which were a huge win on slow windows not so much on linux but we got dragged along anyway. In a similar manner, full virtualization is really slow to deploy and spin up and spin down, containers are smaller and weaker but much like threads vs processes are much faster to start up or shutdown. A really low latency or fast spinup spindown virtualization tech would probably wipe most demand for containerization. Or what I'm getting at is, spawning processes on windows was super slow, so we got threads, and spawning full hardware virt is slow, so we get containerization aka chroot++

octix5• 12 years ago

Every time I see a post like this, I see lots of folks are super exited, but I still fail to see benefits, perhaps is the nature of my/our architecture/setup.

So, let's say I have a fairly typical setup, 1 web app, 1 database. Normally I would have 2 VM instances to start with. When I need to scale I add more nodes for web and/or db... both are load balanced, of course.

How would docker help me? I guess I should google for docker use cases, but I thought I'll try my luck here too.

Test/dev environments are pretty close to production ones.

It seems there is a lot of hype, but I fail to see it through. I'm wondering if all this hype is artificial, time will tell of course.

Thanks.

Why is Docker the new craze in virtualization and cloud computing?

Related Stories