How I migrated from Heroku to Digital Ocean with Chef and Capistrano

http://matteodepalo.github.com/blog/2013/03/07/how-i-migrated-from-heroku-to-digital-ocean-with-chef-and-capistrano/

by matteodepalo • 13 years ago

100 34 13 years ago

matteodepalo.github.com

thibaut_barrere3• 13 years ago

Interesting article I think (as someone who already uses Chef).

That said like I commented on your post, create a proper user instead of running as root, really.

matteodepalo7• 13 years ago

I totally agree and it's next in the list of improvements I want to do. I just wanted to share this asap until it's fresh in my mind.

Edit

Done! :-)

thibaut_barrere3• 13 years ago

Sure :-) I will definitely forward this to people new to this, I would have loved to have such a tutorial a couple years back!

evaneykelen• 13 years ago

I'm planning to move away from Heroku too, partly because of the routing system issues you mention in your article, but also because I want to sharpen my sysadmin fu. Have you considered https://www.cloud66.com/ because it seems like a nice tradeoff between going "bare metal all the way" and Heroku's comfy environment.

matteodepalo7• 13 years ago

I've tried their service a few weeks ago, but they had a problem related to my particular Gemfile so I abandoned it for the moment. Their product is great and I'm sure they will iron out these small bugs, but for now it's not a valid option for me.

zagi2• 13 years ago

DigitalOcean recently became fully integrated with Cloud66.com just last week. It's a great way to easily spin up a new server, and echo's our mantra of simple cloud hosting.

neya• 13 years ago

Cloud66 is awesome I should say, they take the pain of having to do all the groundwork yourself, which is so freakin awesome ! Are there any similar providers to cloud66, apart from Heroku? (and Engineyard ofcourse)

pplante• 13 years ago

I love the idea behind this software. I just wish it worked with Django.

danso• 13 years ago

This was a very helpful and practical tutorial. I don't know why sysops is a source of perpetual confusion for me but it's good to have as many examples as possible. Could you elaborate on the actual hoops that creating a non-root user was creating for you?

matteodepalo7• 13 years ago

First of all I didn't know whether to provision my machine with root or not. Scraping through sysadmin resources I found people installing Ruby with the "deployer" user and people using root for that task. When I tried installing Ruby with deployer I got many permission errors, but I realize that was caused in large part by my lack of experience with sysadministration.

Also, running processes with runit as non-root requires extra [care][1] and I just wanted to get everything up and improve my scripts AFTER I knew that my setup was viable.

Edit

One more thing: I couldn't install ruby-shadow, which is needed to handle passwords with chef, due to some errors with the version of ruby I had.

[1] http://smarden.org/runit/faq.html#user

rschmitty• 13 years ago

With rbenv you can install ruby directly to your deploy user, no sudo required as it installs to your home directory

https://github.com/sstephenson/rbenv/wiki/Deploying-with-rbe...

thibaut_barrere3• 13 years ago

You can deploy the infrastructure with root (or a sudoer), and run the app as a regular user (which is quite often called "deploy" in capistrano recipes).

pw7• 13 years ago

Did the same thing 2 days ago! And cannot stress enough how rewarding the experience was. Although I never heard of Digital Ocean so I chose linode, but I'm also very happy with them so far.

kawsper• 13 years ago

Linode have stackscripts, so you actually could do without Chef with a well structured bash script.

I have asked Digital Ocean about the possibility for Stackscripts, and I don't think they have planned for it, sadly.

It is nice to order a server, choose a stackscript, and have a server to deploy to without even logging in on the instance.

matteodepalo7• 13 years ago

Author here, I updated the article; now a user called "deployer" runs the app. Turns out it was easier now than when I started with the migration.

genericsteele• 13 years ago

This is great. Two weekends ago I did the same type of migration using Linode and Capistrano. Heroku taught us what we should expect from a platform, and it seems like, as a community, we're moving towards getting those great features ourselves.

zagi2• 13 years ago

This is one of the fundamental differences between a platform and an infrastructure provider. As a platform you build a single tool that is used by everyone and with it come inherent limitations. If an infrastructure provider can give you the building blocks to create a platform service on your own, you maintain full control. So long as your time investment into growing the infrastructure doesn't outweigh the cost savings of a platform you get much more predictable growth, better pricing and maintain full control allowing you to develop a product that's customized to your needs exclusively.

purephase• 13 years ago

This is great. Thanks for sharing. I've been toying around with mina, but I may give Capistrano another shot. Right now I have a bunch of build scripts that would be much better suited for Chef and your setup closely mirrors my own.

fomb• 13 years ago

So how does this resolve the Heroku routing problem for you? You have one server that internally routes efficiently - but what happens when you scale past one server?

If you're not scaling past one server, why does Heroku's routing cause you issues?

matteodepalo7• 13 years ago

On Heroku I was using thin that forced me to scale horizontally to handle more than one request at the same time. With this setup I can set a number of unicorn workers bigger than 1 and I'm done. I can scale vertically until I can and then set up more machines with the same methodology and use a load balancer like HAProxy to manage requests.

neilmiddleton• 13 years ago

Why not use Unicorn on Heroku?

mark_l_watson• 13 years ago

He can use a 2-core VPS with plenty of memory and he will have much more headroom than a single Heroku dyno.

How many people really have to worry about scaling past a single server? To me, the issue is more about uptime so that where PaaS's get extra credit as far as I am concerned. It is great to be able to go on vacation and not much worry about admin hassles.

grey-area• 13 years ago

Using unicorn on a vps is the equivalent of many 'servers' or dynos on heroku, and unicorn is taking the place of the heroku routing here, so this does address the problem as you'd have more control over things like timeout and routing strategy, and have other options too for load balancing. If you do need several servers there is software out there to do load balancing, but with caching you can easily scale a single vps to serve a lot of users. For example HN still runs on one server I think, there's a pic of it somewhere that pg posted...

rmoriz• 13 years ago

With https://github.com/rmoriz/knife-digital_ocean you can launch and bootstrap an instance using knife in a single command.

Works with knife-solo and traditional chef-server environments.

JacobJans2• 13 years ago

I would be very leery of hosting any serious business on Digital Ocean. Their terms of service are simply too strict.

For example, you can't send email to your users without double opt-in.

Yikes!

Not to mention the ambiguous language banning 'offensive' and 'inappropriate' content.

mitchwainer2• 13 years ago

Hi - It's Mitch from DigitalOcean. We are still ironing out our TOS. Given that we are still a startup we are always constantly looking to improve the language.

We don't consider ourselves too strict. As long as you are not trying to run an attack or spam/flood our network, we are totally cool. :)

JacobJans2• 13 years ago

Mitch,

I appreciate the sentiment, but I can't stake my livelihood on promises made on an internet forum.

I want to be a Digital Ocean user, but I'm going to base that decision on the legal agreements first.

I do hope this will encourage you guys to remove some of the ambiguity in your TOS, and to remove that double opt-in requirement.

Best,

Jacob

jeromewilson• 13 years ago

Anyone who's interested in migrating from Heroku go do a lot worse than giving Cloud 66 a try. All the lovely deployment goodness without the lock in.

aglosson• 13 years ago

Every time I see someone doing something cool with Chef, I think of this: http://progopedia.com/language/chef/

And then I think, "Wow. This person must be a mad genius."

humanspecies• 13 years ago

I hate the term cloud computing and all the stupid hype that comes with it.

It's called mainframe computing, the exact same clusterfuck your grandfather used in 1970. With bigger hard drives.

How I migrated from Heroku to Digital Ocean with Chef and Capistrano

Related Stories