HN user

justingood

13 karma

Senior Ops at notonthehighstreet.com

my public key: https://keybase.io/justingood; my proof: https://keybase.io/justingood/sigs/bU5WuVQ0JmROZ_E0UBUQawSJ8BqdkLWhMakHTYG3hvw

Posts0
Comments18
View on HN
No posts found.

This is exactly the conclusion we arrived at. We re-build our base images nightly (including all available updates) and then base the application images on those.

We've moved to using Hashicorp's Consul-Template (https://github.com/hashicorp/consul-template). Ansible populates Consul with any required configuration changes during the deployment of a new version, and Consul-Template knows about these changes and automatically writes them to disk. Applications running on the host are then reloaded to pick up the changes.

We're running all the containers on Mesos hosts, so really all Ansible needs to do for us is talk to Marathon. We realized early on down this path that to accommodate scale we'd need to have some sort of scheduler. Mesos happened to be the most robust.

We originally tried the docker module in Ansible but found it had a few problems. There's been a lot of work on it since, and I expect it will be in a much better state when Ansible 2.0 is released.

Ansible 2.0 should have some new strategies to speed things up, depending on your requirements: https://docs.ansible.com/ansible/playbooks_strategies.html It will be interesting to see how performance is after it's released.

We eventually settled on having Ansible build an AMI for us that can then be spun up by as part of a Cloudformation template (also initiated by Ansible).

We've actually been moving further and further away from having Ansible handle the configuration management side of things, and deal with Orchestration primarily.

I think one thing to keep in mind about Ansible is that it's an orchestration tool that also does configuration management. We've integrated Ansible into our workflows in such a way that it kicks off everything we need to do, even if that involves just coordinating some info between APIs. We don't mutate containers at all - merely get Ansible to make things happen around their deployment and communication.

I can second this recommendation. One thing to note: The small folding version is less resilient than the larger model. A fine tradeoff, given its size.

"The technology of deployment does not change 'minutes or hours' into 'days or months'"

I wouldn't say that's true. We're transitioning into multiple languages, and want to have an environment that will allow future languages to be added as required. Building a generic infrastructure to run containers lets us run everything on the same base platform. Otherwise, we'd need to tailor the images and configuration for the individual language type. When a new language is introduced, it can take 'days or months' to get everything working well.

That's not to say Docker doesn't require the same attention to security as other options. This seems to me akin to running a downloaded base VM image without first doing updates.