Web development performance inside and outside a VM

https://www.stackmachine.com/blog/web-development-on-a-vm-is-it-slower

by bdon • 13 years ago

79 43 13 years ago

mmgutz• 13 years ago

Interesting. I switched to VirtualBox for Linux images since it was faster on my benchmarks. Windows was faster and more friendly (drag n drop) on VMware.

bluedino• 13 years ago

Choosing the wrong chipset in VirtualBox can easily create a 50-100% performance difference.

Everlag• 13 years ago

Please correct me if I'm wrong but in virtualbox all I see is two chipsets, one of which is explicitly advised to avoid unless needed.

whitehat2k92• 13 years ago

I've been using a similar workflow for years. My host OS is Windows 7 x64, which gives me all the strengths that Windows is known for with regard to multimedia, gaming, and high-end software compatibility (e.g. Photoshop.) For dev work I run Ubuntu using VMWare Workstation. It's very fast and I get to have the best of both worlds.

conroy• 13 years ago

I run a similar setup. OS X for Photoshop, gaming (don't laugh, Steam on OS X is passable) and day to day, and then do all my development in VMs using Vagrant.

purplelobster• 13 years ago

I run Win 8 on a Surface Pro now with the rest of the setup same as yours. Running Windows gets you the benefit of near-perfect hardware compatibility. Run Linux on a new laptop, and there are always issues. I just don't have the patience to fix it all. It's not always the usual stuff like wifi drivers, now it's more stuff like power management. In a VM, everything just works perfectly.

sbjustin• 13 years ago

I run Windows 7 using VMWare Fusion on my 15" MBP Retina. It's extremely responsive and I can't tell it's running in the background most of the time. I love to use open source but VMWare Fusion is a great product and these numbers support it.

perryh2• 13 years ago

VMWare Fusion/Workstation/etc is easily the fastest type-II virtualization software out there.

Oculus• 13 years ago

I was poking around the website and can't seem to understand what they're selling/offering. It seems like online hosting for VagrantFiles. That can't possibly be it. Anyone know what it is they're offering?

bdon7• 13 years ago

Author here. We're building a service to make dev/prod parity and deployments easy. Thanks for the feedback - our landing page needs an overhaul :P We're still adding features, right now all you can do is generate Vagrantfiles and AMIs.

You can contact me at hello@stackmachine.com if you'd like more details.

JeremyMorgan• 13 years ago

I run Win7 in a virtual machine on OSX for .Net development, since I don't need hardware layer specifics it works great. If I were building applications for the desktop I wouldn't likely it do it this way, but for Web dev it works great.

I installed them as dual boot at first but all the driver issues started getting annoying. So I created a VM and it works great now, plus I dropped a couple Linux distros on there as well.

WestCoastJustin3• 13 years ago

To anyone wanting to learn more about Vagrant via a screencast, I've created one @ http://sysadmincasts.com/episodes/4-vagrant

pwelch• 13 years ago

Did not know there was a sysadmincasts.com. Thanks for sharing.

WestCoastJustin3• 13 years ago

Thanks, I just created it a couple months ago. Found I was constantly researching/learning new things as a sysadmin and thought I would share my notes. Helps me remember too ;) If you have any episode ideas please shoot me an email [check my prfile].

29decibel• 13 years ago

It would be nice to add Docker as a another VM to compare.

whitehat2k92• 13 years ago

Docker isn't really a VM...it's more of a virtual application platform.

IanChiles• 13 years ago

But the best thing about docker is that it's not a VM. It's a containerization layer built on some linux kernel (only) features. It's not fair to compare it to a VM, or even to put it in the same class of programs.

ef4• 13 years ago

To repeat the test with Docker you'd need to use VirtualBox anyway.

(Docker is Linux only. To run a Docker container on OSX, you need a Linux VM.)

shykes• 13 years ago

But you will only need 1 VM :) For realistic testing of a multi-component stack, performance gains start adding up very quickly. There's also the start/teardown time if you want to run tests from a clean environment repeatedly.

In short: the right way to use VMs is as dumb chunks of hardware. Using them as logical units of software makes increasingly less sense as container-native tools get better.

(disclaimer: I'm the Docker guy :)

knotty66• 13 years ago

Anecdotally, Using NFS with Vagrant / Virtualbox instead of the default Virtualbox shared folder speeds up things drastically. Add the following line in your Vagrantfile to test it out:

config.vm.share_folder "/vagrant", "/vagrant", "./", :nfs => true

Also, tweaking the DNS settings seems to help:

vb.customize ["modifyvm", :id, "--natdnshostresolver1", "off"]

vb.customize ["modifyvm", :id, "--natdnsproxy1", "off"]

bdon7• 13 years ago

Author here. Good point, i'll repeat these measurements with NFS instead and add that to the blog post.

EDIT: Updated the post, NFS erases most of the difference between VirtualBox and VMWare Fusion. Nice!

joevandyk2• 13 years ago

Did you test vmware fusion with shared folder?

runamok• 13 years ago

Just an additional data point: NFS does not work when you use vagrant with windows as your host OS. Things like git status when run from within the Ubuntu VM was something like 25 to 50 times slower (IIRC 7 seconds vs. 5 minutes) then when I ran it from git bash in windows.

clippit• 13 years ago

Actually git status is the most annoying part when I'm using Vagrant with Windows. At last I could only use git in Windows to commit my code.

rizalp• 13 years ago

Windows use SMB to share files and folders to the network. We can mount that shares using CIFS. There's already some data posted a huge benefit when using CIFS instead of Virtualbox Shared folder

http://www.wilsolutions.com.br/content/cifs-virtualbox-share...

martinml• 13 years ago

Anybody knows if there is an alternative to NFS, VBox's shared folders and Samba? The first doesn't exist in Windows, and the other two are slow and buggy.

WestCoastJustin3• 13 years ago

Doesn't look like it. The Vagrant docs only list the default /vagrant mount and NFS [1].

[1] http://docs.vagrantup.com/v2/synced-folders/index.html

casey_lang• 13 years ago

In my experience, OSX's implementation of NFS is buggy as well. It's not uncommon for my NFS server to simply give up and stop responding to requests from the vm. Trying out Fusion's share has been on my todo list for a while. I'm a little sad to hear its performance is poor. I've also found that processes like Guard which watch files for changes aren't properly notified. Even with these issues, moving all my development to VMs created by Vagrant has been incredibly useful.

bhauer• 13 years ago

Very interesting reading. Some of my colleagues work with Vagrant or just-plain-old VMs for development and I have heard quite a bit of grousing about performance. It's nice to have some actual data.

For whatever reason, the matter that I found most fascinating is the application start-up time.

I chiefly work with Java web applications deployed on Resin. Some ten-odd years ago, when I first selected Resin as a web container, I did so because of its relatively quick start-up time. It's around 3 seconds to boot a modest application on modern hardware. Since then, I had lost sight of the importance of application start-up time. Resin and many competing but similar platforms (e.g., Play) support dynamic reloading during development, so a full container restart is fairly uncommon.

But your article makes me wonder if it would be of interest for us to include container and application start-up time in our framework benchmarks project. I think that's something we could fairly easily measure.

Thanks again for putting this together!

bdon7• 13 years ago

I've found application boot time to be a pain point if you're running your tests frequently - I run mine several times a minute.

For Ruby/Rails developers it's been especially bad, especially in JRuby. Some major patches have been made to Ruby(MRI) recently to help make this less of a problem:

http://www.rubyinside.com/ruby-1-9-3-faster-loading-times-re... http://blog.tddium.com/2012/11/20/profiling-ruby/

shanemhansen• 13 years ago

I’m using my relatively dinky 2012 MacBook Air - it has a Core i5 chip, an SSD, and 8GB of memory.

Slightly OT, but since when is 8gb of ram plus an SSD plus a quad core machine "dinky"?

bdon7• 13 years ago

Hm, I was comparing it to a typical workstation machine in terms of raw specs - this is my personal laptop :)

merlincorey• 13 years ago

When your production machines have 32GB of RAM, dual SSDs for OS and caching, dual terabyte drives for data, and dual octocore processors.

omgtehlion• 13 years ago

32RAM? man, it's 2013 already

CitizenKane• 13 years ago

The core i5 is a ULV part and it's dual core. Not quad core. Comparatively speaking it is a little bit "dinky" versus desktop parts.

I run a Mac Mini with a quad core i7 and 16GB of RAM. Beyond that consideration, desktop parts are almost always under less thermal stress then those on a laptop and can typically run at higher clocks.

dschiptsov• 13 years ago

This shows nothing.

The more appropriate way, as it was described before, is to take any real-word public data set, import it inside Postgres, then make an app with your favorite stack to lookup and display a set of entries on the page. For more real-world example - add authentication and provide lookups only for authorized users. Then do simple web-server benchmark no these pages.

The improvements with NFS is due to redirecting lots of read operations for OS components form HDD to NIC which means different PCI device, different interrupt, different set of ports. Of course, it helps.

Cheap sharing tenant setups with one cheap SATA HDD for all will never perform well, no matter what virtualization one is using.

Thoughtful partitioning of I/O requests between different PCI devices will help, as old-school system administrators always did, but what is the point of having virtualization then?)

bdon7• 13 years ago

Author here. Great suggestions, however I'm not interested in measuring performance in the conventional page-load sense, but instead the impact of running typical developer tasks in a VM, namely app boot time and tests.

tszming• 13 years ago

I am using Samba and have good experience in term of performance and usability, you might consider adding it into the list.

klibertp• 13 years ago

As a life-long FreeBSD user and a laptop owner I have to run my development environment in a VM. FreeBSD is a great piece of software, but it's hardware support is a bit... lacking.

From the article I got the impression that the main bottleneck is sharing files between VM and a host system. Probably, I can't know for sure because I don't do this. Instead I have all my files and software on the VM, including my IDE of choice. I have X server set up on Windows host (Xming) and when I really need to transfer files between a host and guest I do this over s(f|c)p.

It probably helps that my IDE is Emacs :) (Before that - gVIM) And I don't see any problems with it. Moreover, the only development related thing on the host system is Xming and VirtualBox, so when I change the computer I'm working on I just need to copy the image and install these two programs and I'm ready to go - with exactly the same environment everywhere. I used this setup from Linux, too - the only issue I had was that xhost alone didn't work for allowing external X connections, I had to edit startx script and remove the 'no tcp' (or similar) flag from there.

At work, where I have a proper desktop with nvidia graphics, I have FreeBSD installed as a main OS. The difference in speed between my environment in a VM on a laptop (with 1GB RAM allocated) and on a host system on a better hardware (with 8GB RAM available) is not that big, which makes me believe that I found an optimal way of working. I don't know how more costly virtualization makes disk IO, but I think that compared to general slowness of disk IO this difference is negligible (I don't have SSD yet, so my opinion may change in near future!).

Web development performance inside and outside a VM

Related Stories