Packer is a tool for creating identical machine images for multiple platforms

http://www.packer.io/

189 52 13 years ago

mitchellh14• 13 years ago

Hey everyone. Packer guy here. I've been working on Packer the past couple months and I think its time to bring this to the public. I'm happy to answer any questions you have.

I want to stress that this is the _first public release_. There is a lot of work to be done and a lot of improvements to be made. It may not be "production ready" but it is definitely "experimentation ready." Although I know many companies already starting to use this in production.

I also want to note that although I made Vagrant as well, Packer is not tied to Vagrant in ANY WAY. Packer stands on its own as an extremely useful tool. However, Packer can be used to make Vagrant boxes, as well.

ENJOY!

RyanZAG• 13 years ago

A tutorial showing every step from start to finish for creating something simple, such as a webserver running a database with some custom html pages would be really great.

The docs look good, but it's hard to see which steps to follow in what order for someone who hasn't tried to automate this kind of thing before.

mitchellh14• 13 years ago

The intro actually has a decent step-by-step getting started guide: http://www.packer.io/intro I welcome any feedback!

xorgar8312• 13 years ago

How does packer compare to boxgrinder?

druiid2• 13 years ago

Sadly I would suggest not comparing it to boxgrinder. That project has stalled (I think they actually confirmed so in the mailing-list). Probably a better comparison would be against Oz/imagefactory (the proposed 'replacements' for boxgrinder).

tom_usher• 13 years ago

This looks great, nice work.

For the dev/prod parity use case, where do you usually draw the line between Packer and Vagrant? I'm currently building boxes with Veewee with all dependencies pre-installed and distributing this to the team. Vagrant could easily deal with this, installing dependencies when the VM is launched using provisioners.

Does anyone have any preferences/reasons to go one way or another?

mitchellh14• 13 years ago

It depends on what works best for you. In big organizations, I prefer to take the Veewee approach (although in this case replacing Veewee with Packer). If you use Veewee, see: http://www.packer.io/docs/templates/veewee-to-packer.html

tlrobinson• 13 years ago

Great stuff! I hope this means we'll see a bunch of identical prepackaged Vagrant boxes for all these platforms soon.

mitchellh14• 13 years ago

Yes, yes you will. The Vagrant boxes are now being built with Packer (but not uploaded yet).

TallGuyShort• 13 years ago

When you're configuring the image, is the network stack usable? I currently use Boxgrinder, and the most frustrating limitation is that I can't start and interact with network daemons to initialize things that way, because it's really just a chroot jail (in the Linux sense). Consequently, I have to isolate the embedded databases of the services I'm configuring and install them directly as files.

mitchellh14• 13 years ago

Yes, everything is usable.

AjithAntony• 13 years ago

On your high level descriptions of your tools, like packer, and vagrant, when you say "VMware" you really need to start being clear about which products you specifically support.

mitchellh14• 13 years ago

It is intentionally vague as I intend to support all of them. I apologize for the confusion, however.

geetarista• 13 years ago

For anyone coming from veewee, there is a tool for converting templates to packer: https://github.com/mitchellh/veewee-to-packer

mitchellh14• 13 years ago

And more user-friendly docs for that: http://www.packer.io/docs/templates/veewee-to-packer.html

NHQ• 13 years ago

neat!

csears• 13 years ago

Can Packer build Windows images?

mitchellh14• 13 years ago

It is missing a single crucial feature, but yes it will be ABLE to. The next release. It can currently build OS X machines fine though.

bpatrianakos• 13 years ago

I could totally use this! The only question I have is can you exclude certain things. I'd want to clone 95% of a production server but then would need to exclude SSL settings and such.

mitchellh14• 13 years ago

You choose what you want in the final images, so you can exclude whatever you'd like. You basically have complete control over what gets installed, it doesn't hold your hand in any way, really.

tmzt2• 13 years ago

Is it possible to build AMIs offline? Why is it starting an instance for the source AMI?

mitchellh14• 13 years ago

It is explained in the opening paragraphs here[1] but I'll explain.

There are two ways to make AMIs: EBS-backed or instance-store. For instance-store, you basically need to debootstrap a chroot or something like that. You can do this offline, technically. EBS-backed you just launch a source AMI, build on that, then re-snapshot it.

For the purposes of a 0.1, Packer currently only supports the EBS-backed approach. This is not a technical limitation, however, and in a future version I do want to add the ability to make instance store.

[1] http://www.packer.io/docs/builders/amazon-ebs.html

tmzt2• 13 years ago

Okay, I guess the question was is it possible to pull down a source AMI without starting an instance, then build the new AMI based on that.

ksec• 13 years ago

Written in Ruby as well?

Edit: Just Checked GitHub, it is Written in Go, Interesting..... Another sign of Ruby being painfully slow.

nwjsmith• 13 years ago

I doubt this is why Go was chosen over Ruby, it would be fast enough for this problem. It probably has more to do with distributing the software.

Distributing programs in written in Go is _much_ easier that distributing them in Ruby, i.e. cross compile and ship the binary, but distribution of Ruby programs is hard.

Too much time is wasted on 'omnibus' projects to distribute Ruby programs, just look at Chef or Sensu or Mitchell's own Vagrant installers project:

https://github.com/mitchellh/vagrant-installers

(I don't mean to say those projects were a waste of time, but that it seems like this should be a 'solved' problem).

mitchellh14• 13 years ago

Ding ding.

There are other reasons but this was a _big_ one.

timmow• 13 years ago

This seems to me to be an issue that there is some disagreement on between devs and ops people - most devs are happy to distribute software with a Gemfile and see it as a solved problem. Ops packages tend to go the omnibus route like Chef has, or use Jruby and package as a Jar file like http://logstash.net/

I wonder if there is anything developers can do as a community to help solve the problems these distributors feel they have?

StavrosK5• 13 years ago

Okay, can someone please explain Packer/Vagrant/etc to me? The way I currently provision is using ansible, I have a script that goes from bare machine to serving the site in one command (http://www.stavros.io/posts/example-provisioning-and-deploym...).

How does something like Packer/Vagrant help me? From what I understand, all they do is create a VM, which just takes two seconds in VirtualBox anyway. There must be a benefit, since everyone is using them, but I'm missing it. Can anyone please explain?

njharman• 13 years ago

How does something like Packer/Vagrant help me?

Totally depends on what you need doing. Might want to check out use cases in the packer documentation.

There is some overlap in all these tools. ThAnsible/Salt/Chef/Puppet tend to used to manage "live" OS's/systems, keeping them updated, deploying new builds to them. Although some like Ansible are more generic.

Vagrant and Packer(I know little) manage images. Starting, stoping, provisioning (and often running one of the above Configuration Managments to finish customizing. Although some believe in running from pure AMI, updating them instead of the live systems).

They get more power when there are many devs, lots of images. I.E. when you have enough complexity you need help managing it.

If your small or simple (no negative connotation implied) they probably aren't worth it. I'm one man shop and use Ansible cause it's the right fit for that (although it's expanding fast).

StavrosK5• 13 years ago

Hmm, so what they do is generate an image, "freeze" and deploy it? I'm not very big on that philosophy, I prefer having a script to take the system from a clean state to the current state at all times, but I guess that's just preference.

Maybe it's just that I don't have a big enough need for it, but I did see people using it for single VMs/servers, so I was wondering.

Thanks for the explanation! Also, have you seen ansible's pull mode? I just saw it the other day and I am itching to try it, it looks phenomenal.

johnbender• 13 years ago

I'm not familiar with ansible so I won't attempt to compare the two but I can offer you a short list of benefits that Vagrant provides over building VMs manually with Virtualbox, VMWare, etc.

1. Single command setup across virtualization platforms

2. Tight integration with provisioning tools (ships local config to VM and runs)

3. Network configuration*

4. Automated shared folders (includes NFS since VBox has really bad shared folders)

5. Generally homogeneous config across host and virtualization platforms

6. Safe cleanup on command cancellation (ie, very safe/sane command line behavior)

There's much more to add but this is nice short list. [UPDATE] Generally speaking it takes the idea of using VMs locally to guarantee a development environment/state and streamlines the process enormously.

* Most people underestimate the number of ways that you can screw up network config on local VMs.

zimbatm2• 13 years ago

The combo of point 2 and 4 is what makes the real difference IMO. Since it's easy to provision a new box with a single command and you don't maintain state on them, you get disposable VMs.

No more going trough a list of VMs and asking yourself if you left any work on that or if you can scrap it. Just `vagrant destroy` and `vagrant up` when you need it again. Plus the provisioning bits gets tested a lot more.

1_player• 13 years ago

Off topic: please make your site responsive. I really wanted to read your blog from my phone but the current layout makes it impossible.

Anyway, thanks for that article, very informative.

StavrosK5• 13 years ago

I know, it's bad on mobiles, I'll have to change the whole theme :/ I have started to grow tired of the old one, I want to get a cleaner one. I'll try to make some time for that, thanks for reading, I'm glad you enjoyed it!

zeckalpha• 13 years ago

Not to be confused with https://github.com/keenerd/packer

Nux• 13 years ago

Surprised not to see support for KVM.

druiid2• 13 years ago

Sounds interesting for sure. I don't see any built-in ability to generate qcow2 style images though. Are you going to add that? There are ways to convert (like from a virtualbox image), but having the built-in ability to create qcow2 would be great. Even better would be having a keystone/glance connector to upload generated images to Openstack!

mitchellh14• 13 years ago

Yep, definitely.

nusbit• 13 years ago

May I als you why you choose golang, I am just interested im your opinion.

lojack• 13 years ago

I'm curious where this fits in with my Vagrant/chef workflow where I load a virtual environment with vagrant and provision both development environments and production environments with chef.

Seems as though there's some overlap between how I provision environments and where this tool fits in. What would be the advantages moving this away from my provisioning tool and towards Packer?

joemaller1• 13 years ago

This is seriously ambitious! Glad to see it, a tool for cloning existing boxes was the first thing I went looking for when I started using Vagrant.

wnevets• 13 years ago

would I use this instead of vagrant or with vagrant?

mitchellh14• 13 years ago

On its own or with Vagrant. Packer is useful on its own but also integrates well with Vagrant if you use it.

jimmcslim• 13 years ago

Can Packer meaningfully work with Docker.io? I gather there is some overlap, but is there a way they can work together? It seems to me:

* Packer abstracts creation of images.

* Vagrant abstracts running images, via a VM, and applying changes to images (via chef/puppet, although that seems to overlap with Packer as well).

* Docker seems to abstract both creation of images (via docker build) and running of images (albeit only on top of LXC).

To complicate things further, it seems like Vagrant has some initial support for running (Linux) images on top of LXC (via Fabio Rehm's plugin) and possibly on top of Docker (although the state of the plugin for this is vague).

All this stuff seems very powerful, but the overlap is confusing. My head hurts...

zimbatm2• 13 years ago

Packer mentions continuous delivery in the intro documentation[1]. Is there a scenario where you install the result on existing systems or do you rolling deploys with new VMs all the time ? I'm specifically thinking of services that need to maintain state like databases where the second scenario is not very handy.

[1] http://www.packer.io/intro/use-cases.html

julnepht• 13 years ago

Is there any PXE boot support with Packer ? How does Packer compare to Cobbler + Koan ?

Packer is a tool for creating identical machine images for multiple platforms

Related Stories