Intro to Docker and how we use it at WriteLaTeX

http://jdlm.info/ds-docker-demo/

by jdleesmiller • 12 years ago

157 20 12 years ago

jdlm.info

plg• 12 years ago

q: what did you use to generate the slides?

edit: I see: cleaver

jdleesmiller8• 12 years ago

They're generated by cleaver: https://github.com/jdan/cleaver

You can see how it's set up in the github repo for the slides: https://github.com/jdleesmiller/ds-docker-demo

jonalmeida• 12 years ago

Another option is Brian Cavalier's slide deck format which is similar and easy to use.

https://github.com/briancavalier/slides

errordeveloper• 12 years ago

Each compile job gets its own short-lived container

Just curious, how do handle input and output? Do you prepare a volume with the input and then grab output from there as well, or the program inside the container is fetching input from somewhere first, runs exec latex and then uploads the output elsewhere?

jdleesmiller8• 12 years ago

Yes, we use a volume to get files in and out of the container, and then we upload the relevant files to Cloud Files for the client.

xtrumanx• 12 years ago

Yay, the first Docker post I can understand. I just installed and used Docker today to get a simple web server running out of a VPS (which is also a first for me as I always used managed services like Heroku).

However, some aspects of Docker leave me with concerns that it may not be the tool for me. I tried installing the official node image and it downloaded hundreds of megabytes of other images (probably over a GB). Not that much of a problem on my VPS but absolutely unusable anywhere else in my corner of the world.

Looks like I'll have to create my own images and use my own private local registry to make use of Docker outside of my VPS.

jdleesmiller8• 12 years ago

I'm very glad to hear that it was understandable!

I can't say I've used many of the community images. We generally just use the base OS images and install packages as needed. The Dockerfile makes it pretty easy to do.

incision• 12 years ago

Yeah, those are layers in the image. That image [1] (727MB) is built on buildpack-deps [2] (695MB) which is itself built on the base jessie (154MB).

It would be pretty straightforward to adapt those Dockerfiles to create an image which includes only the dependencies for building node. It would end up looking a lot like 'node:slim' [3] (288MB).

Ideally, Docker will eventually have the functionality to more easily strip out transient requirements like build dependencies from the final images.

1: https://github.com/docker-library/node/blob/013858ac35afb9ca...

2: https://github.com/docker-library/buildpack-deps/blob/a201b1...

3: https://github.com/docker-library/node/blob/013858ac35afb9ca...

qsymmachus• 12 years ago

Really nice presentation. Thanks for sharing this!

rcchen• 12 years ago

Great presentation, it was very easy to follow and understand.

Quick question though: is the only reason for using Docker containers with LaTeX file compilation for providing isolation between documents? Isn't there a performance hit versus running file compilation directly on worker machines, perhaps with some sort of folder-based isolation (workers will only compile files in folders that the user has permissions to)?

jdleesmiller8• 12 years ago

Thanks!

Isolation is definitely one of the main benefits for us. Compared to e.g. a chroot, docker also lets us disable networking and restrict memory etc. for the process in the container. It's another important layer of security.

Another benefit is that the Dockerfile also makes it a lot easier to manage installation of all the LaTeX packages, fonts and various scientific software that we have installed.

The overheads seem to be very low --- less than 100ms extra startup and tear down time, and no significant difference in runtime speed.

frequentflyeru• 12 years ago

Love the format of your presentation. I wish all web based presentations were like that.

j_s• 12 years ago

Docker as a layer of security vs. untrusted user input being compiled as root?

I'd be interested in hearing more about the decision-making since Docker doesn't claim to provide protection from untrusted users.

jdleesmiller8• 12 years ago

Running untrusted programs as root in a docker container is definitely not recommended (and indeed you should be very wary of running anything as root). In the slides, I didn't have time to go into how to run processes in containers without root privileges, but docker does provide features for doing so, and it's definitely worth using them.

bmh100• 12 years ago

Would you please consider modifying or extending this presentation to include other basics? You did a great job at making Docker finally understandable as a whole. I'd love to see more of your writing on this topic.

hackerboos• 12 years ago

I disagree with running RVM in a docker container. The point of RVM is to manage multiple versions of ruby and a container is only meant to have a single responsibility so running multiple versions of ruby is not advised.

I think you're better off installing the required version with Ruby Install [1] rather than adding the complexity of a version manager.

[1] https://github.com/postmodern/ruby-install#readme

jdleesmiller8• 12 years ago

That's a fair point. I guess I recommended RVM by default, since it's what I usually use. Pull requests welcome! [1]

[1] https://github.com/jdleesmiller/ds-docker-demo

adiM• 12 years ago

Would it be possible for you to add support to other TeX formats, in particular, ConTeXt?

jdleesmiller8• 12 years ago

Yes, we are planning to support ConTeXt on writeLaTeX. I think you can vote for it (or add it) on our feedback forum: writelatex.uservoice.com.

Intro to Docker and how we use it at WriteLaTeX

Related Stories