Docker at Lyst

http://developers.lyst.com/devops/2014/12/08/docker/

by Peroni • 12 years ago

91 23 12 years ago

developers.lyst.com

zwischenzug3• 12 years ago

"Dockerfiles are great for simple cases but need a lot of external help to use in more complex situations. Even simple variable expansion would be helpful."

Interesting that many who use docker "for real" have the same problems we did. This is why we ended up building this:

http://ianmiell.github.io/shutit/

There's a long thread on this here:

https://groups.google.com/forum/#!topic/docker-user/3pcVXU4h...

mpdehaan2• 12 years ago

Here's a nice way to do Docker builds with Ansible, which allow using the template engine and all of that good stuff:

https://github.com/ansible/ansible-docker-base

(Disclaimer: wrote Ansible)

zwischenzug3• 12 years ago

Thanks! I may use those as templates to improve this:

https://github.com/ianmiell/shutit/blob/master/library/ansib...

renaudg• 12 years ago

Very nice Michael !

As a longtime Ansible user who's just starting out dockerizing our apps, I find that there's significant refactoring needed when trying to reuse existing playbooks & roles to provision Docker containers.

In the current model we have a single playbook driven by an EC2 dynamic inventory, and each server definition under "roles/" (with some reuse between them, obviously). All in single repo.

With Docker it seems like this layout needs to be turned completely on its head, because a Dockerfile (unlike an Ansible playbook) can only define a single container type, and to make things even harder there can only be one per directory (something similar to "make -f" would have been useful)

So, the logical directory layout in this situation is probably one directory + Dockerfile per container, with a corresponding Ansible playbook + role in each. That's awful for code reuse and quite a bit of refactoring though.

Has anyone managed to do it more elegantly ?

__alexs2• 12 years ago

I attempted to do something like this when figuring out our build problems but it didn't really work for me. It doesn't solve the problem of actually getting configuration into the build very well because it still relies on Dockerfiles and has the added side effect of making the cache mostly useless. Maybe I was just doing it wrong?

I also attempted to try Ansible for deploying containers and had issues with both of these bugs:

https://github.com/ansible/ansible-modules-core/issues/27 https://github.com/ansible/ansible-modules-core/issues/188

Do people actually use Ansible to control Docker in practice? It would be great to have something that was a step up from building random Python scripts from scratch :)

(Disclaimer: I'm not very good at Ansible.)

fndrplayer132• 12 years ago

I'll repost my question from the blog here (And thanks again Lyst team!):

Really enjoyed the write-up, Lyst team! Thanks for the info.

As a member of another large-ish Python dev shop with a number of dependencies, one of things that have kept us from moving fully in the direction of Docker is total deployment time. Since Docker is a 1-process/1-container model, our dependency sizes can be large since we run a number of processes on a single host. This means that our network into a newly launched host would be theoretically very large. How did Lyst deal with that? Or was the flexibility and slight slowness just preferable to something like shipping pre-built AMIs?

boffbowsh5• 12 years ago

We (hi, I work at Lyst too!) see Docker images as fairly similar to pre-built AMIs conceptually, just a lot more lightweight. We bake our Docker host AMIs relatively regularly and use them in the ASG that runs the Dockerized website. Building AMIs (and booting new instances from them on each deploy) takes a lot longer than building a new Docker image and launching containers from it.

I'm not entirely sure I understand what you're asking though. We've so far only Dockerized the main Python application, and not many of the supporting applications. We don't think we'd go down the route of running Postgres, ElasticSearch, Redis etc under Docker.

To give you an example of times taken, Pull Requests take about 10 minutes to test, a full master build (including uploading build artifacts) takes around 15 minutes, and a deployment to 8 Dockerized web hosts is about 5 minutes, including a smoke test on each host.

EDIT: Just realised what you're asking. If you're running many processes on each host then yes it can take a while to deploy. We deploy our Celery workers under separate Docker containers and this can take a while to stop and start. For the website though we run uWSGIs in forked mode, so it's only one container per host. This will probably change in the future, we're evaluating Mesos + Marathon for a more PaaS-like deployment where we can say "14 web processes please!"

Hope that answers your question.

fndrplayer132• 12 years ago

Thanks so much for the answer!

ecaron• 12 years ago

It is amazing how many in this community are able to just dismiss this "Oh yeh, boot2docker performance is still awful." If Docker was a product by Google or Microsoft, we'd have pitch forks out by this point. Yet somehow Docker is still able to skate by on having incredibly subpar Mac OS X performance.

"How Docker Got The Apple Community To Love It, Despite Horrible Performance" <- will someone please write that post, since that's what I really want to know.

__alexs2• 12 years ago

It really has been one of the worst parts of deploying Docker in development actually. I would have gone into more depth on that but every error feels like a personal suffering. We have quite a few developers and many are off site and have odd hardware. You'd think a VM would hide this but integrating with VirtualBox has an ongoing an non trivial internal support cost for us.

tedchs• 12 years ago

If I understand correctly, Docker's performance on OS X is a function of running in a Linux virtual machine, because Docker requires Linux. Is there something about Docker that you feel like inhibits performance beyond the VM itself?

shykes2• 12 years ago

One area of possible improvement is that boot2docker's default mechanism for sharing volumes with the mac is virtualbox guest additions, which are notoriously slow. As for performance, to my knowledge it's simply a matter of the underlying vm config.

tldr: boot2docker should expose ways to leverage faster vm configurations.

andrewguenther• 12 years ago

Do you run OSX in production?

boffbowsh5• 12 years ago

No, 99% of our devs run Macbooks

tracker1• 12 years ago

I gave up on boot2docker, and just use an Ubuntu image with docker as my virtual dev environment.

Even with sub-par performance in development, that doesn't mean it's the same in production... your local desktop isn't going to run a full database instance with equal throughput to the dedicated cluster of servers either, so we should all abandon cassandra (or whatever) too?

arthurcolle• 12 years ago

Are Lyst and Lyft friends?

boffbowsh5• 12 years ago

We haven't yet had the pleasure.

jonatron• 12 years ago

Hello Lysters from the side of the square to your left! Can you give an example of what you replace in your Dockerfiles using Makefiles? We haven't needed to modify ours dynamically.

boffbowsh5• 12 years ago

waves. We don't change anything in the Dockerfiles, it's more that we use the dependency logic in Make to ensure that x is done before y and before z. We also use it to actually run a lot of our containers with the ability to switch in configs for production-like databases if the developer needs it.

Make was the perfect tool for the task at the beginning, but it's getting a little unwieldy and will probably need rewriting in something else soon, probably Python.

Docker at Lyst

Related Stories