Working around the EC2 outage

http://blog.dotcloud.com/working-around-the-ec2-outage

by jpetazzo • 15 years ago

68 18 15 years ago

blog.dotcloud.com

ulope2• 15 years ago

Having read that (and assuming it's accurate) I really wonder how anyone in their right mind can see EC2 as a viable hosting solution.

pan694• 15 years ago

EC2 is built for scalability and distributed systems. The article seems to give a negative spin on something which is the authors own fault.

In most traditional hosting environments you have e.g. one web server. If this one server goes down your website stops working. A solution would be to have two web servers behind a load balancer. If one web server goes down, the other takes over and your site continues to work.

A lot of people who are hosting on EC2 place all their application components in the Virginia data-centers (because its the cheapest data center for reasons the article points out). If the Virginia data center is down nothing works any more. However, EC2 gives you the option to distribute your website over multiple data centers in case of an event like this.

If you choose not to take advantage of this architecture you're no different than running your website on a single web server. E.g. a single point of failure. With EC2 you have to ability to set up a website that never goes down.

Of course, distributing your web site over multiple data centers can be costly. But I guess it's pick and choose, not bitch and moan.

jpetazzo2• 15 years ago

Well, US East (North Virginia) is actually 4 datacenters. As pointed out in the article, multiple availability zones (=multiple datacenters) are affected by the outage, meaning that even if you deploy on multiple datacenters, you can go down.

Now, if you're talking about deploying to e.g. both US East and US West, I totally agree: it would be a good thing to do. But EC2 does not give you that option - not easily, at least, because there is no convenient way to move volumes or snapshots between regions.

Setting up HA between close datacenters (e.g. 50 miles from each other) is easy, because the latency remains low. Setting up HA between datacenters coast to coast is a whole different story, and the only help brought by EC2 is the fact that you can use the same API to deploy your machines here and there.

samstokes• 15 years ago

You're confusing AWS regions with AWS availability zones. You're not technically wrong - it is possible to have your application spread across regions and doing so would have protected against this outage - but doing so is slow and expensive, and the actual failover to a different region is difficult. Amazon explicitly recommends that if you distribute across multiple availability zones within the same region, you should be robust to the majority of outages, which should only take out one AZ. The current AWS outage is affecting all AZs in the US-East region, which Amazon claims should never happen.

This earlier post [1] (HN discussion [2]) discusses this in more detail.

[1] http://justinsb.posterous.com/aws-down-why-the-sky-is-fallin...

[2] http://news.ycombinator.com/item?id=2471899

ulope2• 15 years ago

Well ok that's understood. But still - instances randomly crashing is not acceptable under any circumstances in my view.

shykes4• 15 years ago

With EC2 you have to ability to set up a website that never goes down.

I find it hard to believe that you have any practical experience to back your claim. Today's incident affected sites that scrupulously respected all HA best practice.

sbov• 15 years ago

At least according to the blog post, it seems like part of the issue may be a stampede of requests due to the complete outtage of a single AZ. If every single service hosted in the Amazon cloud did some sort of solution where they did failover to another Region in the case of multiple AZ outtages, how can we be sure that that stampede of requests wouldn't take out that other region? Could Amazon handle the entirety of the US East workload being dumped onto Europe/US West?

snehalpatel• 15 years ago

Choosing to be on AWS or any other cloud provider means you accept some risk of things going down. Build to fail and when that fails, it all comes down to whether or not you can do it better and how much cost you're willing to bear to get to your goal of HA. For me, I know AWS can do a better job at hosting than I can and can accept multiple AZs going down.

mbailey• 15 years ago

Quick answer: have presence in both the VA and CA locations.

shykes4• 15 years ago

The corollary is: ignore the AZ feature entirely. You may be right, but that's a big hit to the attractiveness of AWS.

Working around the EC2 outage

Related Stories