Be Careful When Comparing AWS Costs...

http://aws.typepad.com/aws/2012/02/be-careful-when-comparing-aws-costs.html

by bluemoon • 14 years ago

163 57 14 years ago

aws.typepad.com

benologist3• 14 years ago

This was disappointingly dishonest: they shave the AWS numbers down to a few grand under the self-hosting cost and end on a note saying these analyses should be done apples-to-apples, but neglect to point out the far more expensively flawed methodology in the self-hosting cost: buying and colocating 131 dedicated servers based on the peak requirement of 131 leased EC2 virtual servers.

For those interested it's Amazon's rebuttal to the story discussed here: http://news.ycombinator.com/item?id=3580273

mthreat• 14 years ago

Agreed. Here's a good relevant blog post by mark maunder challenging the "cloud hype" - http://markmaunder.com/2011/10/31/clouded-vision/

I love some aspects of AWS, but I find it hard to trust them when it comes to the cloud vs. dedicated hosting discussion.

efalcao• 14 years ago

I don't understand why the methodology is flawed: the objective isn't to price N servers. The objective is to serve all the load at the best price. If you need 131 during peak and 20 during off-peak and can not pay for the delta- yay!

My company works in the media space and I love EC2- during the week our loads are low but when it's Superbowl Sunday or the GRAMMYs, the ability to spin up capacity is huge. But the fact that I'm _not_ paying for that capacity 24/7 is even bigger.

pbreit2• 14 years ago

How is it dishonest? It is Amazon's attempt to configure the necessary infrastructure to meet the GigaOm post's needs.

benologist3• 14 years ago

It's dishonest because they adjusted their price to be under the self-hosting price, then cautioned that these studies should be done apples to apples. As if they'd just corrected it. What they neglected to do was point out that the self-hosting analysis focused on buying hardware instead of leasing as you do with AWS.

Here's why it sucks:

131 x amazon extra large: $56k after Amazon fixed the pricing

- 524 amazon cores

- tons of crap io

- 2tb of ram

131 x bought servers: $61k

- this includes colo and bandwidth and amortized cost of buying

- excludes labor which Amazon makes mention of

131 x $289 leased servers [1]: $37k

- 524 real cores, each with hyperthreading

- tons of dedicated disk io (raid 1 SSD and raid 1 SATA!!)

- 2tb of ram

- server breaks someone fixes it

These dedicated servers are obviously going to shit all over EC2 virtual servers if you're doing anything that involves disk or cpu so you could in all likeliness shave another $10k - $15k off the hosting bill to match the AWS capacity.

[1] https://hivelocity.net/cart/configure/13c6f96e41#50a2e-82f3f

cmer• 14 years ago

EC2 is about 10-20 times more expensive than dedicated hosting. Even if reserved instances save us 22% over 3 years, it still doesn't even come close. Reserved instances also force me to commit to using a certain type of EC2 instance for 3 years with lots of money upfront. Moreover, IO is ridiculously bad on EC2 and there's nothing that can be done about it. Even RAIDing a gazillion EBS volumes together doesn't do much.

I'm glad the truth about how expensive EC2 really is is starting to come out. Hopefully it will force them to revise their pricing, or at least offer more processing power for the same price.

I ran some benchmarks a few weeks ago that show how expensive EC2 really is http://blog.carlmercier.com/2012/01/05/ec2-is-basically-one-....

That said, I love the flexibility of AWS. RDS, S3, EBS and the whole ecosystem is really well thought out. I just wish I could get real performance out of it.

EDIT: 10-20 times more expensive, not "orders of magnitude"

nirvana3• 14 years ago

I'm sorry to see you're being down voted. For one of my comparisons, Amazon is orders of magnitude more expensive (because I get bandwidth for free as part of the dedicated server package while Amazon charges... even though the dedicated server is both cheaper and significantly more performant than EC2 instances.)

You made a good argument, made some good points, and you linked to a blog article with real data, so naturally you're getting down voted.

I think its a shame that people apparently fall of the "amazon is a tech company" propaganda. I've worked there. It was like working for walmart. Its not like working for Microsoft (worked there too.) The Amazon hype machine is very strong, but people don't seem to realize it, and the amazon hype isn't strictly honest. (EG: They claimed AWS was "running Amazon.com" at a time when none of Amazon.com was running on AWS. They "leaked" the kindle fire to build hype, and brag about how its selling so well, but never release honest numbers, only bundling the fire with all the other kindles in the few numbers they've released. Lots of puff pieces clearly orchestrated by PR firms trying to paint bezos as a visionary, etc. etc.)

pestaa3• 14 years ago

> trying to paint bezos as a visionary

I read about Bezos in Google+ (it was by Steve Yegge, I suppose) and from that, it seemed like he is a jerk but more importantly, a visionary, not unlike Steve Jobs.

What's your take, if I may ask?

Duff• 14 years ago

Amazon is expensive when you do an apples to apples comparison to other hosting options at a small scale.

It's value starts to appear when you are operating at a scale high enough that you start hiring people to futz with servers.

Enterprises love Amazon for a few reasons: - Cost containment -- all costs for a projet can be borne by the project, and go away at the end. - Cost savings -- Amazon costs are easily a third of what large enterprises charge back for similar services. - Avoiding internal IT challenges -- Amazon gives people who own applications or businesses to avoid many of the bureaucratic hoops that they must deal with internally. (Remember the story about the NY Times guy who digitized all of the back issues of the Times? Getting the resources for that project would take months the normal way.) - Speed -- Physical server provisioning times as long as 8 weeks aren't uncommon. VMs can take days. Amazon obviously doesn't have that issue. - Performance isn't an issue -- EBS sucks, but so do the overcommitted VMWare clusters and shared SAN at a large company. At least it doesn't cost $1.50/GB/mo (benchmark cost for enterprise SAN a couple of years ago.)

For a startup that can run the business on a half dozen boxes, it is definitely more expensive. But... if you are expecting lots of growth, you do need to make sure that you choose a colo/hosting place that has the ability to scale up as well.

newhouseb2• 14 years ago

The cost that everyone fails to recognize is the cost (performance-wise) of virtualization. Simply comparing X instances of EC2 to X instances of bare metal ignores the fact that a bare metal instance could range anywhere from 1x to 50x more performance for equivalent specifications. For memory bound applications you might get equivalent performance for the specs but for IOPS-bound or CPU-bound you'll probably take an order of magnitude performance hit for the same specification of hardware when virtualized.

Any analysis like this needs to first find a set of comparable servers that have performance parity (for the given application) first instead of merely specification parity.

nknight2• 14 years ago

If your virtual machine's CPU is 1/10th the performance of host, there's something incredibly wrong. Almost all CPU instructions in a VM run unmodified on bare metal.

otterley• 14 years ago

That's true, but nearly every virtual server provider oversubscribes the physical hosts; there's not a 1:1 correlation between virtualized and physical CPUs. Consequently, your virtual machine may have to wait a while for its process to be requested.

asharp2• 14 years ago

You are correct in that you need performance parity for two items to be comparable.

You are incorrect in saying that you take an order of magnitude performance hit when Virtualization, especially for CPU bound tasks [1].

In terms of disk you are slightly more correct, you can lose substantial performance, however it's still not an order of magnitude.

[1] http://blog.xen.org/index.php/2011/11/29/baremetal-vs-xen-vs... (A series of benchmarks showing a disparity of ~1% off bare metal on CPU bound tasks.)

newhouseb2• 14 years ago

A better comparison might be dedicated hardware versus virtualized in EC2 than virtualized on stand alone hardware as EC2 is a shared environment.

I understand that technically there shouldn't be much disparity for CPU bound tasks (because most instructions are translated directly), but our benchmarks show a 40x performance hit for heavily CPU bound, single threaded tasks between a large EC2 instance and an entry level softlayer dedicated box (doing computer vision work). Perhaps it's a caching issue caused by Xen and other VMs sharing the hardware or perhaps its the fact that EC2 is built on older hardware that might not support some of the more advanced CPU features. Regardless, core for core on a anecdotal level, we have seen a stunningly large impact by switching to dedicated hardware.

DanBlake• 14 years ago

AWS should have shaved down the cost of the dedicated unit as well, if we are in cost cutting mode. Those prices would look very different if they just used some plain jane boxes from 100tb.com or leaseweb.

In order for AWS to make sense, you need to utilize the hourly billing. If you use EC2 and leave you server on 24/7, you are going to be paying more for less powerful hardware than you could with dedicated. Yes, they have a fancy datacenter and instant provisioning and cool backups. You are still paying more. Way more in some cases.

*This does not apply for tiny websites. If your hosting bill is 100$ or less a month, amazon is likely fine for you and maybe the most effective cost saver. Once you go above that mark, Its almost always more effective to be on dedicated unless you are one of the very few people who has truely hourly based service (Think, a seti@home where you only crunch data once a month) - Any normal "website" will do better with dedicated.

Finally, to summarize the above. EC2's main, most important, if-you-arent-using-it-you-are-doing-it-wrong, feature is HOURLY BILLING. If you dont constantly switch on and off from different sizes (and you spend a decent amount of cash) then dedicated is likely better for you.

Dont get me wrong, EC2 is great. But use it for overflow. Or use it for the bells and whistles. Or use it because you have more cash and you would rather not deal with the issues and you like simpledb and yadda yadda yadda. Use it for whatever reason you like. Just dont use it because its cheaper. Because its not.

burgerbrain2• 14 years ago

"*This does not apply for tiny websites. If your hosting bill is 100$ or less a month, amazon is likely fine for you and maybe the most effective cost saver."

I was going to say this. Using an US-East micro instance reserved for 3 years is cheap as hell for my screen/irssi and occasional VPN needs. The cost per month is something like the change I currently have in my pocket.

timc3• 14 years ago

It's still not your cheapest option in many ways, but sometimes. Cheap enough is good.

todsul• 14 years ago

Is AWS really that great for startups (under $100/month hosting)? What about Linode et al? AFAIK, AWS is min $70/month to host a regular dynamic website, whereas Linode starts at $19/month. With intelligent caching (ESI, etc), that $19 can handle 500+ req/sec. Or say 20k+ req/sec for static files (i.e full-page cached sites). Sure there are limitations, but I'd imagine Linode/Slicehost/Rackspace (unmanaged) being better for statups who run regular dynamic websites (with little need for serious elasticity). These VPS providers have load balancing, great communities, good doco, month-to-month contracts, etc.

Admittedly, this isn't my field, so I'd love to hear from someone in-the-know about whether Linode is on par with AWS with regard to most startups (who don't need the extra services Amazon provide over VPS providers).

captaincrunch• 14 years ago

AWS is great for start-ups, specially with the one year free tier. Anyone else giving away a free server, with a load balancer these days?

justincormack• 14 years ago

EC2 is much cheaper than the kind of hosting deals that large enterprise signs, which are not flexible either. That is one of Amazon's really big markets.

sunnysideup• 14 years ago

We are in the process moving our dedicated (always on) servers to EC2 as well. Why? Because even so it is more expensive (almost double in some cases), we compensate this extra cost with reduced admin costs and increased developer productivity (need a server clone for testing? Done in a minute!).

I am convinced that the future of web hosting is AWS style.

verelo• 14 years ago

One of the things that frustrates me the most about AWS is that there are getting to be too many options. Don't get me wrong, love being able to reserve instances, i love being able to size them, but having to also gague if my system is heavy v's light utilization on top of all this gets a little annoying.

Maybe i'm being unreasonable? I would say I am (honestly) if it were not for the fact that if you purchase a 3 year medium utilization instance, and suddenly you want to switch to a heavy...DING there goes your 3 year cost again.

We love AWS, but things like upfront fees that we have to pay, which then have to be re-paid if our business changes really bother me.

Reference: https://forums.aws.amazon.com/thread.jspa?threadID=86667

mark_l_watson2• 14 years ago

I think a fair comparison is very complicated and depends a lot on: do you need to occasionally ramp up the number of servers, do services like Elastic Map Reduce work well in your system for occasional data crunching, do SimpleDB and DynamoDB fit in your architecture, does EBS with high durabilty but slower IO performance meet your needs, does having lots of cheap S3 storage with free inner-data region bandwidth save you money?

If enough of these considerations match your needs, then AWS looks great, otherwise look to dedicated servers.

nirvana3• 14 years ago

If those considerations matched my needs, AWS wouldn't look great because I don't want vendor lock in. That's why they're able to charge high prices-- once you build on their platform, you can't easily migrate elsewhere.

I want to save time by not having to learn Amazon specific APIs (which seem to be terribly engineered, cause every time I want to throw a small project that way I find myself in a morass of poor documentation and way too complificated configurations.)

So, I see that really as an added cost.

yummyfajitas• 14 years ago

You are only locked in if you use SimpleDB, dynamo, SQS, etc.

Most of the standard infrastructure is quite replaceable. S3 can be easily replaced by more or less any file storage system (in Django it's 1 line of configuration), ELB can be replaced by any load balancer, ec2 by dedicated servers, etc.

Admittedly, some code changes are required, but nothing architectural (e.g., S3BotoStorage -> RackspaceStorage in django). For anything bigger than a hobby project, it shouldn't be a major barrier.

mark_l_watson2• 14 years ago

The situation is different for me: almost every customer I have had in the last 3 or 4 years has at least partially used AWS so I have already gone through the learning curve.

Your point on vendor lockin is certainly valid: if you really take advantage of AWS services, you are locked in without a porting effort (like AppEngine).

ypcx2• 14 years ago

I would say, everything considered (thus inevitably comparing apples and oranges at least in some respect), hosting options ordered by longer-term cost from lowest to highest:

1. colocation w/ yours or your admin team physical access

2. AWS

3. normal hosting

And I've put AWS on the 2nd place only due to the ability of replacing or scaling your hardware in minutes.

It's good to know that for $11k a year (paid upfront) plus traffic costs one can run their biggest GPU instance on 10GigE connection, or a 60 GB RAM machine on a normal connection.

I think AWS is absolutely unbeatable for startups where you are not sure about the success upfront.

Also, I'm running my Node.js sites on a Micro instance which I have free for the first year, and $16/mo onwards.

ypcx2• 14 years ago

I wish the downvote action here on HN included a mandatory reason textbox where you'd have to write why you have downvoted, and this action itself would be downvotable. So much political bullshit going on here.

ericd• 14 years ago

I would expect that any downvotes are because there aren't many scenarios where amazon is cheaper long term than leased dedicated servers.

jtchang• 14 years ago

I've never contracted out for bandwidth but do they charge you extra if you under commit?

Dobbs• 14 years ago

A commit is just that a commitment to buy "x" amount of bandwidth.

If you have a commit for 500Mbps at $1 a Mbps on the 95th then you will pay at least $500 a month regardless of how little you use. If you end up using 1Gbps then you will pay $1000 a month.

As far as I'm aware there are no extra charges beyond the price for the commit and any MRC (Monthly Recurring Charges) your account may have. Though my boss takes care of that paperwork so I might be mistaken.

nknight2• 14 years ago

I'm generally right in the middle of said paperwork, and you've got it right. The main caveat is that overage cost is often (though not always) higher than the commit.

(The other caveat is I've never seen $1/mbps transit, if you find it, let me know. :))

foobarbazetc• 14 years ago

This is the biggest, most dishonest, load of shit I've ever read by Amazon.

Yes, there are interesting aspects to AWS. But I could host servers at SoftLayer with 10x more power than AWS instances at a 5-10x price reduction.

Even hosting at an overpriced "managed" dedicated provider like Rackspace would run you less than AWS.

X-Istence3• 14 years ago

It's a shame that Softlayer bought The Planet. I used to host with the The Planet at their bargain brand Servermatrix and they used to have some very good deals, these days Softlayer is charging higher and higher prices, pricing them out of my range.

I wish I could find a good DC that sold me unmanaged servers for cheap. I just need someone to keep the power on, the BW flowing (preferably no Cogent) and when hardware breaks to replace it. I'll take care of the rest.

pestaa3• 14 years ago

> I just need someone ... when hardware breaks to replace it

I'm not a sysadmin, so I might be terribly wrong here, but isn't this the very definition of managed hosting?

rubyrescue• 14 years ago

stormondemand?

benologist3• 14 years ago

Hivelocity is cheap: http://hivelocity.net.

We've been with them since 2010.

emmett• 14 years ago

Links, please. I believe that rackspace servers are cheaper/more powerful per box, but an order of magnitude on both sides is absurd.

What you are claiming is that $100/month on AWS would cost $1-2/month on SoftLayer. That's just not credible to me without a citation.

ddon• 14 years ago

We at Fotki were approached several times by Amazon, and they tried to make us switch, and every time we calculated, we were getting around $75..100K per month just for storage, not including traffic and other stuff... It is pretty much true that their price is 10x or 20x of owning your own hardware.

nivertech• 14 years ago

In order to be competitive with dedicated hosting, AWS should introduce per-month heavy utilization pricing option. 1 and 3 years is too long to commit for startups.

fasteddie31003• 14 years ago

One thing I've always thought about the three-year reserved instance is what happens if AWS lowers the price on the configuration, you've already lock in on the higher price. Three years is a lot of time, technology advances quickly and prices decrease, but if you lock in on the three-year contract, you won't see the benefits of lower costs. Actually, now that I write this, maybe this is how Amazon justifies the lower price for longer contracts.

nathanhammond• 14 years ago

About a month ago I did a bit of pricing for having a permanently-on dedicated virtual server through Amazon:

Micro - $105.80/yr (reserved instance)

Bandwidth - $0.00/mo (light transfer usage)

S3 - $0.20/mo (combining storage and transfer)

CloudFront - $0.20/mo (combining requests and transfer)

After a quick glance around I found that it was going to be impossible to beat that price, most especially for the services gained by using AWS (CloudFront, S3, CLI interface + tools, ability to spin up an instance behind the scenes and swap over to it once properly configured...)

The moral of this story is to make sure that you price out the cost of using any platform for your particular usage profile.

rkalla• 14 years ago

I am surprised I don't see the geographic distribution of AWS as a huge appealing aspect of the service.

If you are trying to deploy a global solution, I don't know any way as cost effective as rolling out images to all the regions you need and rolling out the same scripts, knowledge, strategies to each region to get your service up and running.

Opening accounts with dedicated providers in each country as an alternative sounds infinitely more painful.

ridruejo• 14 years ago

We have thousands of users who run their servers on Amazon using one of our AMIs (http://bitnami.org) Whether EC2 is more or less expensive than the alternatives is a complex topic highly dependent on your particular requirements. Having said that, our perspective is that you need to consider the overall value you get for your money. There are features like the ability to take incremental snapshots of your entire machine for backup or cloning purposes that have no match by other hosting providers. That feature alone saves us innumerable hours of development and system administration. Our staging servers are not setup like or production servers, they can be exact copies of your production servers, on demand. How many bugs/issues do you think we avoid with that? For us, enough of them that any price difference with a "traditional" setup is not worth it

okrasz• 14 years ago

AWS is cost effective in some scenarios while not in others. It really strongly depends on actual needs, what resources are needed, in what proportion, etc. There is an article which tries to answer if Amazon is really the cheapest provider: http://blog.cloudorado.com/2011/08/is-amazon-cheapest-cloud-...

You can compare the prices with custom configurations of multiple providers with Cloudorado - http://www.cloudorado.com.

Be Careful When Comparing AWS Costs...

Related Stories