Which do you use: Physical server, or hosting in the cloud?

I'm curious what other Startups are pursuing as a means of hosting their web apps.

I personally choose having a physical server at a location (I typically buy or build the server myself)

The basis of my choice relies on the breadth of configuration and customization that can be done when you have root access to a 100% dedicated server.

As a background: I'm currently working on a Startup that's in the Healthcare IT realm and the web app that we're building is very resource intensive and requires many different packages.

Now granted not everyone is in the same boat and doesn't require that much flexibility but I'm curious how many still do choose to have a physical dedicated server.

Eventually if the startup that I'm working on takes off, I will most likely need to use some sort of elastic hosting (Amazon), but this comes after you see whether your idea has legs and that you can raise funding required to build out the infrastructure required. In the meantime having a physical server cuts down on costs (no monthly recurring costs, just the 1-time fee for the server and $10/mo extra to be able to host off my Internet, and I have a fully 100% dedicated server). Best part about this option is if the Startup fails, the server can be used for the next endeavor and I'm still only paying just $10/mo for dedicated hosting.

Paying for a similar hosting setup that I have now would probably cost me several hundred dollars a month.

Please state your choice/preference below and any pros/cons that you've come across