Replacing a Development VPS with Linux on OSX

http://jmlacroix.com/archives/osx-linux-vps.html

by remi • 15 years ago

91 63 15 years ago

tombell• 15 years ago

Wonder if the author has considered checking out http://vagrantup.com/ unless it does a bit too much for what they're after.

lisperforlife• 15 years ago

+1 for vagrant. It is awesome. It integrates well with Chef for provisioning. I would definitely recommend it.

1880• 15 years ago

I currently use a similar environment but with VirtualBox. Does qemu offer any advantages over it? I was considering recreating my VM so I may as well try qemu.

jmlacroix4• 15 years ago

It's been a while since I last used VirtualBox, so I can't really compare.

What I like in QEMU, is its simplicity. The learning curve is a bit steeper, but when you're used to it there's no turning back.

mise2• 15 years ago

On a tangent to this, what is the "Linuxy" stuff you don't get to do on OS X? (My own frame of reference being an Ubuntu user.)

mhd4• 15 years ago

The author might have a different experience, but for me it's mainly anything that a) depends too heavily on a plethora of GNU tools and b) anything that's about deployment and installation. You're basically running on a different Unix system, so if you deploy to a Linux server, you'll probably encounter quite a lot of differences - mostly deployment, but also libraries, syscalls etc. Quite reminiscent of the days of different commercial Unices, where the compatibility of e.g. AIX and HP-UX wasn't exactly 100% (I'm sorry if that caused some violent flashbacks for veteran developers)

Also, OS X is usually behind even compared to Linux distros with fixed releases (i.e. not rolling). You get your basic Unix software in one fell swoop, with the occasional update through a dot version. Apple doesn't provide a pkgsrc or even a rpm/deb packaging system. There are third party efforts, like MacPorts, Fink or – as the author mentioned – Homebrew, which is more similar to BSD's package tree or Arch's AUR (basically a git repo of Ruby build scripts).

Nevertheless, these add on to your system, so aren't really integrated perfectly. Also, compared to Debian or Arch, you'll have a smaller package list (esp. true for Homebrew). So it might just be easier to install some stuff from within a headless Linux VM.

Heck, I've been doing something similar even when I was running Linux as a dev environment. Your deployment environment might be quite different, and I like to do some local testing first – doubly so if your company doesn't even have a proper testing/integration environment. I had an Arch PC, but deployed to both somewhat recent Ubuntu machines and ancient SuSe installations. Now, I often work on OS X and deploy to a CentOS VPS. All good use cases for VMs – and the way described, where it's just a small headless instance in the background instead of a useless GNOME desktop-within-a-desktop is a pretty good approach.

jmlacroix4• 15 years ago

Hello, author here.

This is really a great overview of why I like this setup, I couldn't have put it more clearly.

An example of something I've been trying lately and didn't work well under OSX: ruby bindings for FUSE filesystems. I wanted to write a quick FUSE fs using ruby, so I installed gems and other dependencies. But since MacFUSE has been deprecated for a while, it's not easy to get everything working.

It could probably work under OSX, but given the current state of FUSE on Lion, I gave up (and did not try it under Linux yet).

While trying to setup my FUSE environment, I installed every FUSE version, many gems with a couple of ruby versions with rvm, rbenv, ruby-build, etc. When playing with new tools and environments, I always end up with lots of useless or broken stuff.

This setup allows me to keep my base system clean and simple.

I do not use it daily, but it's nice to have it ready to launch when needed.

mise2• 15 years ago

shudder! Suddenly it's not such an attractive thought to invest in a MacBook even for (L)AMP development, and deployment to Cent OS.

cmer• 15 years ago

I also wonder. I don't think I ever ran into a single thing that works on Linux but does not on OS X.

zobzu2• 15 years ago

It's all about the extra pain. (or the lack thereof)

dchest3• 15 years ago

Valgrind on Lion.

LXC (obviously).

losvedir• 15 years ago

It's not necessarily that you "can't" do a lot of things -- it's just harder. Recently I was trying to play around with Xmonad. I managed to get it up and running in X11 on OS X but it always was a little "off". I wanted it full screen, but sometimes the dock would appear, sometimes not, sometimes the OS X menubar along the top would appear, sometimes not. Sometimes various applications would lose focus but stay in front of X11. It was just not very great.

Then I ran into some problems getting gvim working in that setup. I'm not completely facile with all things Linux and usually Google for some help setting things up, but in this case I couldn't find anyone else who wanted to install gvim on OS X using X11, so I was out of luck there.

In the end I just fired up a Linux VM and it was all as easy as could be.

rmc• 15 years ago

Installing anything complicated. It usually Just Works on Debian/Ubuntu, but for OSX you have to tweak things.

zokier4• 15 years ago

Apt with 20000+ packages waiting to be installed.

zobzu2• 15 years ago

OSX is an OS where you run VMs for Linux and Windows. I keep seeing people saying how great and fast it is to do stuff on their mac and run Windows for a few apps incl. mail client, office suites, some dev tools and many java apps (cause they're often not so nice on OSX)

Then I see them with the Linux VM for all the dev tools, the hack tools, and so forth, because, well, hacking them up to work in OSX is always a pain, and even using stuff like brew is often a pain (+ delay)

Well I dunno, I run Linux native and I get stuff done faster than the ones running Linux in a VM of course. I also have Linux VMs under Linux when necessary, with KSM and it's a lot faster and memory efficient than when running OSX native on the same hardware.

I install stuff in about <5s where it takes a VM to boot/resume in OSX, and about 5-10min to install from scratch when the app _supports_ OSX (ages if it doesn't but has more or less compliant sources)

MatthewPhillips• 15 years ago

Yeah, me too. I have a linux computer for most development, a Macbook when I want to mess around with iOS development, and a Thinkpad for .NET development. All of these are tax write-offs and since I use the linux computer (which twice as old as the other two, by the way) 90% of the time because it's simply easier to install development related stuff. Whenever I want to install a gem that uses native headers requiring a -dev package in OSX I just quit.

ubercore• 15 years ago

I'm the opposite. I like all of my OS X dev tools (navicat, textmate/macvim (I know, gvim...)/sublime text/github for os x/things), and it's a much more usable OS overall for me. The parts I don't like, compiling libraries, running servers, etc, I automate away with vagrant. I'm more productive with this setup than if I ran everything directly in Linux (I've tried).

barumrho• 15 years ago

Your points are valid, but running VMs, and using unix tools are not all I do with my computer. For me, running OS X is a good compromise.

I used Linux exclusively for more than four years, and I miss many things about it, but I don't think I can go back, at least yet.

chrischen4• 15 years ago

Running a VM also lets you isolate your dev system and simulate your production server.

pmarsh2• 15 years ago

Unless you're on an ancient laptop I don't see why you wouldn't run a Linux VM if you are developing web apps.

What is the advantage of trying to make OS X your development environment and then deploying to a Linux host?

Unless you have an older laptop with less than 4 gigs of ram why not? Even with 2gigs it's not that system intensive to run a Linux guest os.

riobard3• 15 years ago

Let's face it: SSH into an VM and write code there is not an enjoyable experience. Yes I can do a proper deployment into the VM right after I make a minor change in color of the 10px square box located at the bottom of the page, but it's really a pain in the ass to have the deployment delay. Oh, right, the deployment process just takes 10 seconds!

pmarsh2• 15 years ago

Why SSH into a VM? With Virtualbox I have a folder shared out that I can use like a regular folder and edit with whatever editor on my Host. Or you could turn on SMB shares and use those.

Or I could run a lightweight X environment (or even gnome it really doesn't need much to run) and just edit there.

Everyone seems to be on the "OMG Vim/Emacs are so awesome, TextMate is so yesterday" part of the every going cycle anyway. Might as well use them on the guest OS as well.

olefoo2• 15 years ago

And it's worth running a vm for development even if you're hosting it on a linux machine; presumably your workspace has a bunch of things that don't need to be installed in production.

maratd5• 15 years ago

There is a problem with doing things this way. I want my development server to be always on, always available. It does things in the background and sometimes I use it when I'm away from the office.

Yes, you can keep your desktop machine on all the time, but that's a huge power suck. Shut down your desktop when you're not using it. Most development servers just don't need to be that beefy anyway.

A better approach, if you're absolutely committed to virtualization, is to just spin up an instance on Rackspace or Amazon. You'll pay about 10 bucks a month, which is less than you would for keeping your desktop running 24/7. On top of that, if your desktop dies, you won't lose your data.

dchest3• 15 years ago

Not everybody needs to run development server 24/7. Plus, the title and the content of this post is "Replacing a Development VPS with Linux on OSX", and you're suggesting the opposite.

maratd5• 15 years ago

> Not everybody needs to run development server 24/7.

Great. Shut down the instance. You pay nothing when it's off.

> Plus, the title and the content of this post is ...

I know what the article says. I'm telling you it's not a good idea, for the reasons listed above.

I do run a few local images here and there, but that's only for situations where I don't have access to the internet.

zokier4• 15 years ago

Why would you need to have your dev server always-on? I think you are either speaking of different kind of development server or your dev server is doing too much stuff.

When I hear development server, I think of box on which I can try out various things which may, or may not blow up. That makes it inherently not always available. Also, I want it to be in a relatively clean state so I know that any problems rising are not side effects of some other thing running on it or anything. And after I finish testing some new stuff, I want to be able to just throw it all away if necessary, without needing to care if there are some other services that may be disturbed.

maratd5• 15 years ago

> When I hear development server, I think of box on which I can try out various things which may, or may not blow up.

I'm not always in the office. Sometimes I'm at home. Or on a trip. Or in another country. And I still need to have access to my code, try things, make sure they don't blow up.

I suppose I could just use Wake-On-LAN or IPMI, which I do sometimes ... but my development server also doubles as a file server and collects error logs from deployments. Not kosher, I know, but no $$$ for anything else.

jmlacroix4• 15 years ago

Hello, I'm the author of the post.

These are all good points, I should have made my goals clearer. I do not use it as a web development machine or something that needs to be always on.

I use it as an occasional machine, to compile and test stuff that's not working or unavailable under OSX. I was looking for a powerful and cheap VPS (which is uncommon) when I figured I'd be better served by a headless VM.

Another big plus for me is having access to it when there's no connectivity.

maratd5• 15 years ago

> I use it as an occasional machine, to compile and test stuff that's not working or unavailable under OSX.

Makes perfect sense in that context.

mentat2• 15 years ago

If you're mostly looking for headless VM, you should check out Vagrant. It makes setup, teardown, and provisioning pretty straightforward.

emehrkay• 15 years ago

I recently got into an endless back and forth with a guy on reddit because I said that as a web developer, I require a mac (instead of a windows machine). When asked why I said that it is a hell of a lot easier to mimic testing/dev/production environments on a mac than windows and the only reason to choose a mac over a linux distro is because of the adobe dependance. His stance was that it isn't important to have your machine muddled up trying to replicate the dev/staging/production machines when you simply can develop on a remote box via some sort of ssh/file mounting. To which I replied with not always having access to remote dev machines, etc.

What is your take on this issue? I was actually surprised that people want to develop on remote machines. It reminds me of "ftp to see a change"

shadow_s• 15 years ago

That's how I develop. Since I use vim anyways, I just ssh into a linux dev server from my windows box and code right on the server with the only delay being latency. 99% of what I do can be done in the command line so I end up with few problems.

nkassis• 15 years ago

bah, not really if you use editors like emacs or vim. I'd suggest also using screen to maintain a dev session going. screen lets you start shells and detach from a session without closing any of them (useful for long running jobs too). In the end my remote environment doesn't differ much from my local environment.

technomancy• 15 years ago

I do most of my development on my server at home from my laptop (either at home or at coffee shops), but if I hadn't already invested in a beefy box I would definitely be using http://pair.io instead.

dpritchett• 15 years ago

I develop (Django/Rails) with Chrome running locally and my dev environment running in vim/tmux on a Ubuntu box that I ssh into. As long as the dev server automatically reloads when it detects a file change I see no issues.

What are you thinking you'd miss in a setup like this?

coob• 15 years ago

Being able to work offline.

typicalrunt• 15 years ago

There are some useful tips in that article.

Using qemu in a no-gui mode is pretty cool. I've used VirtualBox, VMWare Fusion, and Parallels on OSX before, but I hate having to deal with the GUI window (and having the system reserve resources for it).

Since OSX is based on Unix under the hood, it shouldn't be too difficult to for a developer to use either OSX or Linux as their development machine. It's not like the knowledge gap between Windows and OSX/*nix machines. So other than messing around with Linux, the article fails to mention why he doesn't use OSX (other than some unfounded 'flaky' and 'impossible' critiques).

ch0wn• 15 years ago

I used to work on Django project on a Mac. Dependencies like python-mysqldb can be quite tricky to install on OSX, whereas Ubuntu only requires you to type 'apt-get install python-mysqldb' and you're done.

More importantly, though, I do not like the idea of having all those services constantly running on my desktop system. I'm using Ubuntu for both development and deployment now and I still use a separate VM.

Bockit• 15 years ago

I may be missing something but I think you can do what is proposed with VirtualBox. I use it for my virtual dev machines and run them in headless mode. For example the command:

vboxmanage startvm "Name of VM" --type=headless

to boot an image up, takes about 10 seconds to be ready to ssh into (You can ssh immediately it just waits till it has booted to connect).

vboxmanage controlvm "Name of VM" poweroff

If you want to turn it off from the outside.

jmlacroix4• 15 years ago

Hello, I'm the author.

I mostly use OSX for everything. The only times I switch to the VM is when I'm having a hard time installing something (for an example, see my earlier comment - http://news.ycombinator.com/item?id=2969509) or want to try a tool that's not available under OSX.

I admit the post would have been better with examples and without then unfounded critiques, taking notes for the next ones.

martijn_himself2• 15 years ago

Just wondering if you could think of anything obvious I'm doing wrong (I'm pretty much a newbie when it comes to this)...

I'm following the instructions in your post on my Macbook Pro (2010); however I get 'Booting from DVD/CD... 180MB medium detected. Boot failed: Could not read from CDROM (code 0005) No bootable device.' I am using the netinstall Arch Linux ISO.

Many thanks!

halostatue• 15 years ago

VirtualBox also can run without GUI windows:

    VBoxManage start vmname -type=headless

If you need to see the GUI, you need the VNC extensions installed (which are, IIRC, not FOSS).

chrischen4• 15 years ago

It's often a good idea to simulate your production server as closely as possible.

zokier4• 15 years ago

I think one of the main reasons why I like VMs so much is that they allow my base OS to stay relatively clean. Especially as I like to try out all kind of new stuff constantly, VMs make that easy without having to care how they affect the rest of my system. Even if you aren't trying out new stuff, keeping separate projects mostly isolated makes me feel happier. imho also configuring stuff becomes easier when your machines become basically single-purpose systems.

mentat2• 15 years ago

If you want no-gui VirtualBox that "just works" try vagrant.

anon_koder• 15 years ago

archlinux!

riobard3• 15 years ago

Anyone knows the difference in performance between QEMU and Virtualbox both running on OS X?

jinushaun• 15 years ago

I also "deploy" to my local Linux VM for when I want to test a multi-server set up or try something new without messing up my main computer. Otherwise, I code and test almost exclusively on OSX. Haven't run into any situations yet where I wished I was coding in Linux as well. Linux doesn't have Photoshop or iTunes.

martijn_himself2• 15 years ago

I'm following the instructions in the post on my Macbook Pro (2010); however I get 'Booting from DVD/CD... 180MB medium detected. Boot failed: Could not read from CDROM (code 0005) No bootable device.' Does anyone have any ideas why this is not working? I am using the netinstall Arch Linux ISO. Many thanks!

itodd• 15 years ago

Protip for those of you virtualizing Linux using VMware Fusion: You can force quit VMWare Fusion and your VMs will continue to run in the background.

Replacing a Development VPS with Linux on OSX

Related Stories