Cluster-Level Container Deployment with Fleet

http://coreos.com/blog/cluster-level-container-orchestration/

by robszumski • 12 years ago

108 15 12 years ago

coreos.com

nl• 12 years ago

I love this.

How are (non-public) Docker images handled (as in where are they stored)?

Do they need to be built on each machine before use?

robszumski3• 12 years ago

If you run a private index (ex. index.mycompany.com) docker will automatically fetch it just like the public index. Protecting this index/registry with authentication is an entirely different beast though...

In a unit file, you would substitute `docker run coreos/subgun` with docker run index.mycompany.com:1234/subgun`. Docker handles the rest as long it can communicate with that network address.

brugidou• 12 years ago

Interesting to see how this evolves in comparison to Apache Mesos/YARN. There are definitely similarities with etcd<->zookeeper, fleet<->mesos, docker<->cgroups...

steveb• 12 years ago

There are a few efforts to support docker (and other) containers in Mesos under (https://issues.apache.org/jira/browse/MESOS-816)

As part of this, a couple of new docker efforts were released today:

https://github.com/tarnfeld/mesos-docker-containerizer

https://github.com/mesosphere/medea

I'm pretty excited to see fleet. These types of distributed systems are a natural complement for containerized infrastructure.

siliconc0w• 12 years ago

You can run docker containers with marathon pretty easily plus other stuff: https://f.cloud.github.com/assets/908362/1405593/a19d12da-3d...

I'm liking marathon, it makes mesos much more accessible and is under pretty active development.

jaytaylor• 12 years ago

I'd like to see it support bare LXC containers. That could be very powerful IMO.

kapilvt• 12 years ago

"User job requests kick off a bidding process by the machines in the cluster. Based on the received bids the engine assigns the job to a machine." yeah.. superficially that sounds very similar to mesos.

another system that can do cluster wide containerization and orchestration across clouds/baremetal is juju.

The end goal for all seems to be in utilization efficiency and ease of management at scale of a host of application topologies.

contingencies• 12 years ago

I see this and think: "another from scratch distributed system without proven reliability/stability". Personally I use corosync/pacemaker for my container orchestration.

chubot• 12 years ago

I've heard of those systems but never knew anyone who used them. Any good writeups or demos to try?

tinco• 12 years ago

Do you guys have any idea how to support docker containers that are going to use persistance (like database containers)? Or is fleet to low level for that and do you expect higher level tools to supply information like that in meta data?

Very interesting stuff!

robszumski3• 12 years ago

We're leaving it up to the user. Ideally this is a replicated database that handles failover automatically, but it can be an attached storage device, NFS share, etc.

morganherlocker• 12 years ago

Note that this is not the same fleet as (also a deployment tool): https://github.com/substack/fleet

ermintrude• 12 years ago

I think the big gap with docker is logging. If you're going down the coreos route you'd probably want your logs centralising somewhere. That means that you need to configure your containers to run something like supervisord and rsyslog to log to a remote server which is a bit of a hassle, especially if you were making use of public images since you'll need to add all this boilerplate in yourself and rebuild the images.

Alternatively I guess you could log to the host if there are config options to log directly to syslog and you could then ship them from there.

I spent a couple of weeks writing a python app and the same again trying to sort out a flexible logging system. Ah well, I guess it's still under development, but it'd be great if they prioritised this since it's one of the major things stopping me using docker (and obviously I'm so important they should reprioritise around me!)

I really wish coreos was bundled with a package manager though. Not having telnet or the ability to apt-get install things easily makes any sort of debugging and real-world use infeasible for me... :-(

robszumski3• 12 years ago

Does the current systemd journal not work for you today? On CoreOS, your app outputs to stdout which should log into docker which feeds logs into the journal which fleet exposes with fleetctl journal myapp.unit.

I realize that this isn't a proper logging product or anything, but it should be enough to hold you over. If this isn't working for you or you have a different use-case, hop in #coreos on Freenode and we can try to figure something out.

jbeimler• 12 years ago

I've been looking into using logstash similar to how it's described here http://denibertovic.com/post/docker-and-logstash-smarter-log...

Cluster-Level Container Deployment with Fleet

Related Stories