We're moving. Goodbye Rackspace.

http://code.mixpanel.com/amazon-vs-rackspace/

by suhail • 16 years ago

203 174 16 years ago

code.mixpanel.com

powdahound2• 16 years ago

Goodbye code.mixpanel.com too? Page won't load. :(

suhail6• 16 years ago

funny part is...it's on the rackspace cloud.

Though probably our fault since we gave it a crappy box.

Edit: 512MB of RAM should be enough...

riffer• 16 years ago

What is the limiting factor here?

I am curious because it seems like almost any configuration & setup should be able to serve a blog post to the volume of request that come with being at the top of HN.

suhail6• 16 years ago

Apache2 basically just OOM'd

redorb• 16 years ago

google-cache-version http://webcache.googleusercontent.com/search?sourceid=chrome...

seldo• 16 years ago

Yeah, not exactly a ringing endorsement of rackspace that the rackspace-hosted blog saying they're leaving is down under the load.

barrydahlberg2• 16 years ago

I'm suprised to see a setup using 50+ instances running on RackSpace cloud, wouldn't it have made sense to start moving towards dedicated servers by then?

cperciva7• 16 years ago

Not necessarily. Sometimes flexibility is worth more than having the minimum possible cost per clock cycle.

barrydahlberg2• 16 years ago

True but in theory the advantage with RackSpace is you can mix and match to balance the cost of your known requirements with the flexiblity for your unknown.

suhail6• 16 years ago

We need the elasticity. I can't wait 24 hours for new boxes to come.

tnm• 16 years ago

Capacity planning. It's a thing. Try it out.

jamesgolick4• 16 years ago

First of all, there are managed hardware providers who can get you hardware online in less than 4 hours. Second of all, capacity planning can save you from "needing elasticity". Third of all, if you were on machines that gave you reasonable performance (i.e. not shared hosting), you probably wouldn't "need" nearly as much elasticity.

benologist• 16 years ago

Yup planning is hard when the next guy that registers can double your load, but dedicateds give you a lot of bang for your buck and you can still use VPSs (or cloud stuff of course) to scale up in a hurry.

I'm pushing a ton of data (>200m events/day now) through 2 dedicateds + 1x VPS and if it's a bad day I get 1 or 2 extra VPSs deployed, each of which adds another x,xxx events per second capacity.

sanswork5• 16 years ago

I have 50+ instances on another cloud provider. The service cost benefit on going to a dedicated server is a fair bit but the management cost goes through the roof. We also prepay to save some expense on the monthly bill.

I can easily manage those servers, spin up new ones as requirements change, drop ones I don't need anymore, etc with very little hassle.

We also have a few dedicated servers(one actually with Xen on it for running smaller instances) and the time and effort to manage instances on it makes it pretty much not worth it for us without a dedicated sysadmin.

powdahound2• 16 years ago

Make sure you explore the limits of EBS before assuming it's a perfect solution. We've found it to have incredibly slow throughput at times.

Some reference links:

  - http://orion.heroku.com/past/2009/7/29/io_performance_on_ebs
  - http://victortrac.com/EC2_Ephemeral_Disks_vs_EBS_Volumes
  - http://stu.mp/2009/12/disk-io-and-throughput-benchmarks-on-amazons-ec2.html

gyardley• 16 years ago

We had these issues as well.

I'd certainly try AWS, but I'd wait a couple of months before shelling out the upfront cost for the Reserved Instances.

blantonl2• 16 years ago

EBS definitely has performance limitations, however creating striped RAID arrays of EBS volumes significantly helps with performance.

This blog post walks through a bunch of things you can do to make the best out of the flexibility you get with EBS, and includes scheduler tuning, RAID block size planning, etc.

http://orion.heroku.com/past/2009/7/29/io_performance_on_ebs...

My team runs a large AWS/EBS backed implementation and, while we've seen some of the EBS performance issues, we've certainly engineered past them to get us where we need to be.

imbriaco2• 16 years ago

As a heavy user of EBS, I heartily concur. The ability to move EBS volumes around is handy but you pay it back in inconsistent, poor-to-mediocre performance. Good luck.

recampbell2• 16 years ago

I've also seen highly variable io performance (ie, slow/stuck) on EBS volumes. Sometimes, all reads/writes block for several second or minutes, with iowait going to 100% on an m1.small.

example: http://developer.amazonwebservices.com/connect/message.jspa?...

However, I've decided that the other benefits of EBS volumes are just to important to give up on (snapshotting, lazy-load, re-attachment). Instead, I plan to monitor for such situations and blow away the node when I detect it.

But really, I don't understand why Amazon doesn't fix this. It's happened to me 3-4 times in a relatively small installation. Surely their monitoring can detect this?

dasil0032• 16 years ago

Is it because of the network throughput on the small instance type? I haven't noticed anything nearly that severe with our 20-instance setup of larger instance sizes, but then again we've only run production on EC2 for a few months now...

moe4• 16 years ago

But really, I don't understand why Amazon doesn't fix this.

When you select your instance-type then you get to choose between "Low", "Moderate", "High" and "Very High" I/O performance.

Want to take a guess at which instance types suffer first when there is high I/O demand or contention?

cperciva7• 16 years ago

Quoth the article: Lastly, we moved over to the Rackspace Cloud because they cut a deal with YCombinator (one of the many benefits of being part of YC).

Can anyone say what this deal is, or is it secret?

varenc• 16 years ago

Just reduced price hosting.

My guess is its a deal that won't be happening again? :-)

mikeyur• 16 years ago

For a separate project I've worked on (non-YC), we negotiated 50% off bandwidth and somewhat reduced server rates. You just need to ask for it.

Not sure what the YC deal is, but I assume it's similar.

aaroneous• 16 years ago

I had a similar experience: with minimal negotiating we were able to get 50% off their initial quote. I also renegotiated our terms later within the contract period with relative ease.

They pad their pricing pretty heavily, so don't be afraid to ask.

mattlong• 16 years ago

Can't speak for other YC companies (it may vary from batch to batch), but for us it was a generous rackspace cloud monthly credit for 12 months.

joecode9• 16 years ago

One potential reason not to move: Security.

A good friend deep in the security community once told me, off hand, that EC2 was "owned." I didn't take this too seriously until another good friend, who has been working at Amazon for the past several years, told me that engineers at Amazon were generally forbidden from using AWS due to security concerns.

That much said, I still decided to use EC2/RDS/S3 to host the infrastructure of my latest startup. It is just too convenient to walk away from. Once it matters, I can move the critical stuff to dedicated servers.

EDIT: To clarify, I'm not suggesting that Amazon knows AWS is "owned" and offers it to others anyway. I'm only noting that, for certain critical services, they themselves do not appear willing to take the risk.

rbranson3• 16 years ago

While this is disconcerting, I wouldn't make any business decisions based on such a claim. The idea that EC2 is "owned" without Amazon knowing about it is closing in on absurdity. I've worked directly with Amazon as an outside vendor and they are very security concious, to the point of near paranoia.

joecode9• 16 years ago

While I agree it is hard to believe, it would be even more surprising if Amazon did know about it. The fact that they do not use AWS internally suggests that---at least with their level of paranoia---they seem to suspect AWS themselves.

dangrossman• 16 years ago

I'd love to hear more about this...

cperciva7• 16 years ago

I've worked with Amazon Web Services security people in the past, and while they're not perfect (nobody is) I have always had the impression that they take security seriously. AWS has many very large customers, including the US government and companies handling HIPAA-restricted data; based on the assumption that Amazon employees don't want to be thrown in jail for 10 years, I think it's safe to say that if EC2 is is "0wned" as you claim, it's certainly not well known within Amazon.

tptacek3• 16 years ago

For what it's worth, accidentally (or even negliently) violating HIPAA is fantastically unlikely to get you charged criminally.

jread9• 16 years ago

I'm not sure there is any proof Rackspace Cloud is any more secure than EC2. AWS offers a Virtual Private Data Center service (VPC) which is highly secure. Rackspace Cloud has nothing like that. AWS also offers firewall management functionality which Rackspace Cloud does not. Amazon.com is run out of the same data centers as AWS.

tptacek3• 16 years ago

The odds of EC2 being owned and Rackspace not being owned are astronomical.

joecode9• 16 years ago

Certainly agree. Perhaps it would be better to suggest moving to dedicated servers, depending how security-paranoid you are, anyhow.

benblack5• 16 years ago

"...told me that engineers at Amazon were generally forbidden from using AWS due to security concerns."

The opposite is the case: there has been a huge push for some time to move (significant) parts of Amazon retail to AWS. It's extremely complex and service quality is paramount, so it takes a while to make it all happen.

joecode9• 16 years ago

My friend from Amazon works in the supply-chain side of things, and he said he really wants to use it, but everything has to be encrypted and some stuff is off limits.

I take it you work on the retail side of things? I'd be interested to hear any more details that you can share.

joseakle• 16 years ago

Could you expand on your comment?

benblack5• 16 years ago

"A good friend deep in the security community once told me, off hand, that EC2 was "owned." I didn't take this too seriously until another good friend, who has been working at Amazon for the past several years, told me that engineers at Amazon were generally forbidden from using AWS due to security concerns."

"EDIT: To clarify, I'm not suggesting that Amazon knows AWS is "owned" and offers it to others anyway. I'm only noting that, for certain critical services, they themselves do not appear willing to take the risk."

I may not be the smartest guy, but it seems to me that's exactly what you are saying.

joecode9• 16 years ago

I'm not sure where the confusion lies, but I'm guessing you see "security concerns" as equivalent to "knowledge of ownership"?

It seems to me those are entirely different things, as one can be concerned about potential threat without knowing if it is real or not. But I do not work in the security community myself and may be using language sloppily.

I would be much obliged if you could show me where the crux of the confusion lies.

dschobel• 16 years ago

That's a hell of an allegation to perpetuate based on hearsay.

yesimahuman• 16 years ago

Another control panel complaint: DNS. For some reason beyond me you have to choose a server just to configure DNS. Added on top of the fact that the control panel is really slow, it just becomes a pain to use.

nl• 16 years ago

Yes!!

I was trying to do a DNS change and I couldn't find the DNS panel anywhere until I Google'd it.

jread9• 16 years ago

CPU is a major bottleneck for Rackspace Cloud. All instance sizes get the same 4 cores and about the same compute resources. CPU performance is roughly the same on a 1GB cloud server as an 8GB cloud server, you are just paying for more memory. Rackspace also uses ONLY Opteron 2374 2.20 GHz processors. EC2 on the other hand offers linear CPU performance improvement on larger sized instances. EC2 also uses a heterogenous hardware ranging from Opteron 2218 or Xeon E5430 for m1 instances; Xeon E5410 for c1 instances; Xeon X5550 for m2 instances and Xeon X5570 (hyper-threaded to 16 cores) for the cluster compute instances. EBS on the cluster instance is also much faster than local disk IO in the Rackspace Cloud based on testing I've done (due to non-blocking 10G network). Here are a couple of references for this:

http://blog.cloudharmony.com/2010/05/what-is-ecu-cpu-benchma... http://blog.cloudharmony.com/2010/06/disk-io-benchmarking-in...

jread9• 16 years ago

<quote>Amazon has a CDN and servers distributed globally. This is important to Mixpanel as websites all over the world are sending us data. There’s nothing like this on Rackspace.</quote>

Actually, Rackspace cloud offers CDN services with Cloud Files through Limelight, although it does not support some features that CloudFront does like CName and streaming.

suhail6• 16 years ago

But not physical servers which is really what I am referring to.

jread9• 16 years ago

Rackspace Cloud has data centers in Texas and Illinois. However, they don't let you choose (your account is assigned to one data center at the time of creation). EC2 certain offers a lot more flexibility in this regard with EC2 data center regions in California, Virginia, Ireland and Singapore. I spoke with Rackspace at Interop a few weeks ago and they told me they are working on expanding to additional data centers including an international data center very soon and offering choice.

justinchen• 16 years ago

A CDN is probably the easiest service provider to switch. We've tried Panther, Voxel, Cloudfront and use Edgecast. It's just a matter of changing some CNAMEs.

adriand2• 16 years ago

This is a rather unpleasant review of a service we were about to move over to.

For companies that have moderate performance requirements (e.g. visitors in the range of 30k or 40k per day across a range of web apps and sites), reasonable but by no means expert level server administration skills, the need for a redundant environment to satisfy SLAs with clients (e.g. two app servers + load balancer + master/slave db servers), and the desire to focus mainly on software development instead of server admin, what companies does the HN community recommend?

We've been considering Rackspace Cloud and Linode, but are open to any suggestions. We also have a quote for a standard, managed four-server + hardware load balancer deployment in front of us but it is pricey ($3000+/month).

sanswork5• 16 years ago

I've used Rackspace, Amazon and GoGrid. I'm currently almost entirely on GoGrid and just finishing up migrating back off of Amazon for the second time(Migrated some services back to them when they opened west coast).

There have been some stability issues in the past year(instances going down for maintenance regularly) but haven't had a problem with that in a while and was advised it was due to them upgrading all the servers in their cluster.

The ability to mix dedicated and cloud systems easily has been the killer feature for me. We also prepay and the number of servers we get for the cost is pretty good.

madaxe3• 16 years ago

Linode are wonderful - best support I've had from anyone, ever. Nice instances, too!

shazow• 16 years ago

+1 to Linode.

catch233• 16 years ago

another +1. we moved to linode from rackspace. They don't have that crazy backup limitation that rackspace does, and they're always open to custom inquiries. Rackspace has the big company mentality -- if our interface doesn't have an option for it, it doesn't exist.

boyter• 16 years ago

What are they like for CPU usage? I have been looking at VPS.NET and other such providers to ensure I get a reasonable slice of CPU which I can use for other things.

Most VPS providers I have come across dont like it if you start working out the millionth digit of Pi, but sometimes you have a task that takes a while and you can't run offsite.

tlack2• 16 years ago

+1. In the past 24 months I've used Rackspace, ThePlanet, SoftLayer, Slicehost, Linode and the terrible Mediatemple. I'm doing all future development on Linode. Their basic VPS is pretty quick for the price and it's online almost instantly. It's still just basic Linux, so you can rig everything you want - no awkward managed interfaces to worry about.

I would consider switching to a different company, but for now Linode scratches my itch better than the rest.

easp3• 16 years ago

I like Linode, but they are wearing a little thin for me.

In general, uptimes have been excellent but the physical server hosting one of our VPSs had three emergency reboots in a little over a month before they replaced the hardware. Perhaps they'd already planned to do a replacement after th e second incident but I had no way of knowing.

Which brings me to my second beef, which is that they don't share much when closing a ticket. Their explanations are pretty prefunctory. In some cases, like transient network issues I generally can't tell if they've investigated the problem and done something about it or if the first-tier tech just takes a quick look sees it's working now, and punts it back to me to let them know if there are more problems.

Similarly, after a major screwup that took down most of the virtual servers they host (including the ones that hosted their own website and web-based management tool), all we got was a brief explanation of what had gone wrong and a status blog hosted on someone elses infrastructure.

What I didn't get was a sense of how they were going to guard against anything similar happening again.

I'm sticking with them for now, but I'm also feeling like the effort I've put into automating the configuration of our application and all it's dependencies has been time well spent. I can boot a new infrastructure on another service in an hour or so and migrate to it with minimal downtime.

MichaelGG2• 16 years ago

Have you looked at just buying the hardware and colocating it? $400 should get you half a cabinet. Bandwidth can be found for a few dollars a meg. As you're buying redundant servers, you can get refurb/ebay stuff and save on the upfront costs (who cares about next business day service replacemnt if you have spare parts?).

3pt141592• 16 years ago

Don't. It isn't worth it. Just use AWS or Rackspace or Linode. Outsource hardware management to people that do it for a living.

lsc6• 16 years ago

I wrote a thing about the pros and cons of doing that, but instead I thought it would be better to give you some objective numbers.

1 full rack with 15a of 120v power (remember, don't exceed 75% of a circuts rated potential, so that's really 11.25a) at he.net in freemont is going to set you back $400/month. $500/month with 100Mbps bandwidth. (more than you will need for now)

Now, you are talking between 5-7 low-power servers there.

You want to buy a metered/switched PDU ($400 or so new, $150 used)

one of my 8 core, 32GiB ram opterons with 4 disks is about $2500 in parts, new. I can't imagine anyone charging more than $500 to put the thing together.

Note, unless you have really cheap hardware guys and downtime costs you little, I'd avoid used hardware. I started on used servers, and I no longer buy used hardware that is user-serviceable. (I avoid used servers. But I may buy used switches.)

If you need racker/stackers I can point you at experienced guys who will be happy to work for $50-$60/hr, and people who can get the job done at half that. If you want more advanced hardware troubleshooting, double that for someone really good. but for that kind of money they should be able to set you up with an automated provisioning system. for comparison, I'll work for $100/hr if you are willing to deal with my inconsistent schedule.

Setting up a server should not take more than a few hours, including hooking up the serial console, etc... And hardware problems? figure on once every two years per server.

a lot depends on the quality of your provisioning setup (something like cobbler is good) and your serial console/rebooter setup. But it /can/ be a lot cheaper than 'the cloud' in many instances.

edit: added the bit about used hardware.

nwilkens• 16 years ago

Softlayer (or AWS) + outsourced server management is great fit for your scenario. Softlayer has great HW / network, and can provide a good base for a scalable infrastructure. AWS EC2 also sounds like a great fit in your environment.

Regardless, having a great server admin guide this implementation will be a key part of the chosen architecture success.

I run a server management company, so I have a bit of a bias ;)

jackowayed• 16 years ago

Depends what your app is written in.

If I were doing this with a Rails app, I'd use Engine Yard for sure. Engine Yard Cloud makes stuff like load-balanced app servers and master/slave replication really easy (you can do it from the web interface), and they really know what they're doing and have tuned the stack based on that knowledge.

I don't know if there are similar offerings that are reasonably priced yet awesome for other languages though.

gommm2• 16 years ago

I found engineyard very unflexible... Sure you can create custom recipes to get around that but since they update their system without warning, the custom recipes end up becoming stale...

schammy2• 16 years ago

I hear lots of awesome things about Linode, however they are actually hosted by Rackspace so with Rackspace's known reliability issues... maybe an issue.

Another service to consider is VPS.net. We have 6 virtual servers with them. They are cheap and perform well, their control panel is great and they have tons of great pre-configured OS installs. However downtime can sometimes be an issue. Usually it's not major, e.g. a server will just be unavailable for say 30 seconds or maybe a few minutes, but this happens probably an average of once or twice a month for each of our servers so it can get annoying. Like I said though the outages are usually minor. There's only been one that I remember that was major, and lasted several hours. We've been using VPS.net for about 8 months and overall are very happy with them. Not sure I'd recommend them for your setup but we only use them for a custom built CDN and for that it works very well.

bombs• 16 years ago

Are you confusing Linode with Slicehost? Slicehost was acquired by Rackspace.

mdgeek2• 16 years ago

linode servers are hosted on The Planet data centers. Check this link: http://serversiders.com/linode.com

mdgeek2• 16 years ago

VPS.NET servers hosted on Softlayer data centers. Midphase,Anhosting & 100TB.com (all in UK2 group) also on Softlayer.

Hostgator,Linode,SiteGround,Site5 hosted on ThePlanet. JustHost on Liquidweb dedicated servers.

Hostmonster on Bluehost servers.

newobj2• 16 years ago

Wow, that's a brutal under-the-bus-tossing. But well deserved. Most of the points are spot-on.

I've operated a several dozen machine fleet for 2+ years on EC2 and I can tell you that the number of times boxes have gone down never to come back (which most people tend to think happens regularly on EC2) is incredibly small.

We actually run a Hadoop cluster (including DFS) on a spot instances. We never lose the spot bid. We pay way less than the going rate for the compute time. Less than the reserved instance rate. It's awesome. Yes, you obviously need a plan to deal with your cluster vanishing in the blink of an eye. It's not too hard.

I would second another commenter's caution on EBS though. We never put it into production. Personally I never experienced an ephemeral drive failure that had repercussions - when they (rarely) occurred, the drive was RAIDed or in our Hadoop cluster (e.g. redundant). We made two experiments into using EBS with our DB's, and both times, literally within 24 hours, we experienced a catastrophic failure of the EBS volume, one time unrecoverable. So that put me pretty well off EBS.

I can't say that the performance is necessarily the best, and we do experience the occasional odd asymmetrical inter-machine latencies (e.g. 300ms to establish a tcp connection in one direction, but normal <1ms in the other), but for the most part AWS is just awesome.

gruseom2• 16 years ago

Yes, you obviously need a plan to deal with your cluster vanishing in the blink of an eye.

How long do the spot instances typically run before they vanish?

cperciva7• 16 years ago

A spot instance should stay around until the auction price is higher than what you bid. That price has been quite stable recently, so as long as you bid slightly higher than that rate you'll probably have your instance for months.

newobj2• 16 years ago

They only vanished once (the entire bid and therefore the entire cluster), and I think it was actually due to a glitch; the price never actually went over our bid. That's the only time we ever lost our spot instances. I think we actually bid over even the normal (non-reserved pricing), because I've (almost) never seen prices cross the reserve price, and even if we had to pay full price for a day, that's fine -- it's still tremendous savings in the long run.

dangrover• 16 years ago

Rackspace cloud's DNS stuff stinks. No way to add TXT records -- you have to open a ticket! Sure you can host it yourself, but every other cloud provider has this in their UI.

I get the feeling they're just in "maintenence mode" over there and don't have anyone working hard on improving the offerings.

danudey4• 16 years ago

Their DNS used to be great, if simple. You could put in a record for your new server and by the time you'd logged in, the record was live.

Now, it's gotten to the point where their DNS updates so slowly that I just re-use old subdomains instead of creating new ones, just so that I can get my project tested today rather than tomorrow.

I'm working on moving everything over to Amazon; more flexibility. I like Rackspace Cloud, but they're falling behind and they either don't know, don't care, or can't catch up.

megaman8212• 16 years ago

What is even more sad, is that their DNS console for hosted servers and Slicehost is so much better. They are literally sitting on something better and giving their cloud customers crap.

lisper• 16 years ago

It's even worse than that:

http://rondam.blogspot.com/2010/03/danger-will-robinson-rack...

xal• 16 years ago

My honest question is why there is this odd loyalty to virtual environments in this community. I realize that it may be boring but you guys are passing up insane savings that can be had by using colocation. All cloud providers are very expensive when you actually do the math and you need more then 10 servers.

Our example may be a bit extreme, but we are just building out a new datacenter at a colocation and will recover the entire up front investment ( about 150k, we have the cash to not need leasing ) in a bit over half a year.

chopsueyar4• 16 years ago

Can you provide some details on your infrastructure?

alttab• 16 years ago

We also use colocation. I'm not a developer on the team that uses the infrastructure but we use OnRamp in Austin, TX (http://www.onr.com/) and I have rarely heard of any issues. Most delays are from international CDN requests which I believe we have also outsourced to speed it up.

spidaman• 16 years ago

There's a big difference between the cost to install and operate and the cost to scale. If your hardware utilization characteristics are easy to calibrate for and capacity requirements are predictably steady, you can certainly optimize your own racks at a colo. However, hardware changes quickly, businesses typically change even quicker and capacity requirements are often highly variable. Unless the above conditions are met, it's insanely risky to go into a contract for a colo. Let us know how that one year investment recovery works out for you.

tdavis2• 16 years ago

Dedicated (rented) offers similar performance improvements and cost reductions without the extreme upfront cost (obviously costing more over its lifetime as a result). But these days it really seems like a blind spot. Maybe it's just that I never completely jumped on the cloud bandwagon (or was running servers far before it got rolling), but I really haven't found a generic use case for cloud hosting. Planning infrastructure upgrades isn't rocket science; you'll have more than 10 minutes' notice to get a new machine. There are certainly many specific use cases for cloud hosting, but as a generic hosting tier I find it incredibly overused—and companies waste money and are forced to deal with unreliable performance (EBS i/o anyone?!) as a result.

ergo98• 16 years ago

GoGrid lets you mix and match dedicated and cloud servers, which offers the performance and value of a baseline of dedicated, and the flexibility of the cloud. I find it to be unparalleled.

sanswork5• 16 years ago

I don't think you can compare every ones case equally though. In my case it would be a lot more expensive to go with co-location even ignoring the upfront costs.

Right now I pay $4K a month for ~93Mb of Ram available per hour. Averaging 2gb per server for web, bit more for caches and application we can fit a lot of servers in our 93Mb.

Could I get a couple beefy colocated servers setup Xen and run my own cloud? Sure, but then I have to pay someone to run that cloud and I have to manage and worry about the hardware and those costs are a lot more for me than a small monthly markup.

There are a few things I don't host in the cloud(master db, and primary load balancers) but even those I pay a bit more to have them be managed hardware wise.

catch233• 16 years ago

I do colocation as well, but if you don't have someone there to babysit your server, sometimes it can be nightmarish. It's nice to not worry about if a disk fails and you need someone to swap out one of your raid spares and hopes he doesn't swap out the wrong drive bay.

With colocation companies, everyone is sending their own custom built servers to the data center so nobody knows the exact configuration per server. If a drive fails, or a memory chip goes bad, you need someone there that knows the layout of your hardware config to debug the issue. I have a bunch of servers at RippleWeb but I'm sure the guy there has to deal with 1U servers from Dell, Tyan, Supermicro, and all kinds of other manufacturers.

schammy2• 16 years ago

Agreed. We do colocation for about 50 servers. Up front costs higher, long term costs MUCH lower - but man can it be a pain in the ass sometimes when there's a problem. :)

retube• 16 years ago

I have to say, for every provider I've looked at, the cost of disk space on virtual hosts seems insanely excessive.

zemaj3• 16 years ago

I moved all my services from EC2 to Rackspace Cloud about 2 years ago, but I'm regretting it.

Rackspace Cloud does one thing well - small instances have great value CPU & local IO performance. If your app is CPU or local IO bound, splitting it across multiple 256MB instances on Rackspace Cloud will get you huge performance relative to price. I've been worried that this would degrade as the service grew, but that hasn't been the case.

Unfortunately many other 'features' of Rackspace Cloud have been poor to awful recently. Some anecdotal stories;

1) We haven't been able to make images of our server or restore ANY backup of our servers for months. There is a bug in the backup service where if you have spaces in the names of your Cloud Files containers (completely unrelated to the backup service) then all images fail to be able to be restored. We can't remove the spaces in the containers because you can't rename containers (only delete) and there's too much data tied to different parts of our infrastructure in there.

2) In relation to the issue above, we have had a ticket open for over 2 months which we continually post updates with new information & asking for issue resolution. We never receive updates to the ticket itself and only receive information when contacting their live chat. The response is always "we're working on it". I could live with it if this was a short period, or not an absolutely vital part of their service, but come on - all backups broken for 2 months! No timelines on resolution. No ticket responses. No happy.

3) While CPU value is great on small instances you get the other end of the stick on large instances as other posters here have said. You don't get significantly improved performance above the 2GB servers. CPU capacity certainly does not double as their documents say.

4) Cloud Files latency is awful. Individual read/writes take 300-1000ms. Fine for a small number of large files. Impossible for a large number of small files. (Having said that, being able to upload files and publish to CDN in a click has saved me lots of time for static files I need to quickly publish).

5) Resizing mid to large instances is impossible. We recently tried to resize a 1GB (40GB disc) server to a 2GB (80GB disc) and it took OVER 3 DAYS. No really. It didn't complete. The resize process takes the server down towards the end. We had to get Rackspace to cancel the resize and manually spin up another server and transfer the files ourselves. To make it worse, we couldn't act on this issue initially because Rackspace insisted that the process was "almost complete" from 12 hours onwards. 2.5 days later we just gave up. We managed to do the manual transfer ourselves in a couple of hours. Even worse Rackspace seemed to not think that it was unusual for the process to take 3 days or express any desire to investigate further.

6) The web interface has awful performance at scale. Once you go above 20 cloud servers every single page load takes 10+ seconds. As the original poster says, the number of errors it spits out about not being able to complete operations is insane. It's rare I can go in there planning on doing something and not have to contact support to fix something broken on their end.

7) They're taking the entire web interface and API offline for 12 hours this week! You won't be able to spin up or take down any of your servers. Why? So they can fix a billing issue related to Cloud Sites (a service we don't use).

I've always been a champion of Rackspace Cloud and Rackspace in general, but sadly I would no longer recommend them to people. I'm starting to make contingency plans and looking for other providers again.

jread9• 16 years ago

Rackspace Cloud used to have the upper hand on smaller sized instances. This is no longer the case with EC2's new 640mb micro instance at about $8/mo reserve. Not top performer, but it gets the job done for lower traffic servers.

zemaj3• 16 years ago

Huh, that's good to know, thanks. That makes my moving choice easier :)

alex12• 16 years ago

Not to mention you could now have an entire year of EC2 micro (and a bunch of other services) for free.

russell_h2• 16 years ago

For anything that needs much CPU at all that isn't going to be the case. Any Rackspace node will be literally an order of magnitude faster for anything CPU bound, compared to an EC2 micro instance.

On the other hand if you just need memory (without necessarily having the clock cycles to quickly access it), or just something that will eventually handle occasional requests, a micro instance should be just fine.

danudey4• 16 years ago

At a previous company, we were using Rackspace Cloud for one of our sites, and we decided to double the size of the server as we were under a lot of load and needed some capacity fast.

Each step completed (the new server was spun up, the old server was backed up, etc.). Then it said it was going to take the server offline to do the actual clone of the disk from one to another. Then it took us to a screen saying 'Did everything complete properly? Yes/No', saying that clicking 'no' would just boot the old server back up again.

We waited for an hour, then two, then three, and the server still hadn't come back online. I didn't want to click 'no' in case it was something simple, so I called their support. After explaining what had happened to the guy, he told me to click 'Yes', which I did. Suddenly, the new server booted up and came back online. Support told me that when you get to that screen, you just hit 'Yes' and then your server boots up.

I couldn't help but think this was the dumbest thing I'd ever heard. They take your server offline, tell you it's going to be back 'any minute now', and when it is, you tell them if it worked properly or not. In reality, your server stays offline until you say 'Yeah, everything's working great', at which point they bring it back online.

I still recommend them to people who need basic services, such as 'I need a dedicated server for my blog'. Amazon is flexible, but also very complicated for semi-pro users to deal with.

zemaj3• 16 years ago

The whole resize thing is fraught with problems. It was one of the main reason I moved to Rackspace Cloud, but it's just way too unreliable. I'm too scared to use it on any mission critical servers due to unpredictable downtime.

If you're under high traffic and you try to resize the server the operation time hugely blows out. There's no way to block this TCP traffic other than changing DNS settings which is obviously undesirable and you might as well just manually setup a new server if you're going to do this (which is what I now tend to do). This process should be automated. It's an obvious bottleneck and fixing would dramatically improve their offering.

I didn't really mind Rackspace having all these issues at the beginning, but after 2 years I see no progress anywhere. Unlike AWS there's very little apparent investment in the infrastructure tools. Other cloud services have matured, but Rackspace hasn't. While the support at Rackspace is awesome, there's only so often I can hear someone apologizing. I'd rather the issues just got fixed and all this friction was reduced. Incredibly disappointed.

jread9• 16 years ago

Another distinguishing factor between Rackspace Cloud and EC2 or Linode are bandwidth caps. Rackspace limits outbound public network throughput to 10mbps for 256mb instances to 70mbps for 16gb instances. EC2 and Linode both provide an uncapped GigE Internet uplink for instances of any size. http://cloudservers.rackspacecloud.com/index.php/Frequently_...

megaman8212• 16 years ago

This actually becomes really important when you set up a small server as a proxy sever. We had to bump up our HAProxy server to a 512mb instance in order to get adequate bandwidth to serve our sites.

jfine2• 16 years ago

They both limit internal network speeds also. EC2's limits are much higher than RS's.

mikey_p• 16 years ago

I'd also add that their billing software can't keep things straight. I've had a couple of servers that I spun up for a demo at a meetup, that somehow ended up with the same name, this prevented me from deleting them, and tells me that their control panel has concurrency issues, since you shouldn't be able to create two servers with the same name.

I didn't notice the issue until 2 months later (I thought I had successfully deleted the servers) because all of the sudden I received huge bill that contained over 1100 hours of usage for each instance, for that month alone. WTF? Turns out their software failed to bill me the previous month so I didn't notice any change.

Their response to my ticket about not being able to delete servers was to tell me the steps that I had to take to fix it (renaming the servers). I really wish when you had a ticket for stuff like that they'd actually act on it instead of just telling you how to fix stuff and expecting you to do it yourself.

We're moving. Goodbye Rackspace.

Related Stories