Ask HN: Max no. of EC2 instances used by a single web app?

https://news.ycombinator.com/item?id=1450187

by jm3 • 16 years ago

23 12 16 years ago

news.ycombinator.com

Including DB & API machines, our web app occupies 18 heavily utilized Amazon EC2 instances under a traffic load of ~7k RPM. "Conventional wisdom" says that we're mis-using EC2 by keeping so many virtual instances running but I prefer hard data to "conventional wisdom".

Curious to hear from the HN community: what’s the most long-running EC2 instances you've seen in a successful web app?

jm33• 16 years ago

I’ll go first: we (140 Proof) actually have 20 instances, not 18. How many instances do other people’s apps use?

kehunt• 16 years ago

Some other data points:

* Animoto was running several thousands of machines in 2008 (http://bit.ly/EDLtt)

* Litmus runs 400 servers (http://bit.ly/d7Hc7y)

* 99Designs runs entirely on EC2 (http://bit.ly/aotKgg)

I've personally had long-running EC2 instances with uptimes in years. You could do it cheaper in terms of hardware, but at the cost of wasting time at the colo while you could be building cool shit.

nethergoat• 16 years ago

Good start - I wasn't aware of the Litmus environment. The Animoto example is great, I love telling that story to people just starting to look into cloud computing.

To add to the large-environment roll call, here are the persistent server counts of some 100% cloud-hosted companies:

- Bizo: 100+ instances (I work here)

- Reddit: 100(?) instances ("256 Virtual CPUs" http://us.pycon.org/2010/conference/schedule/event/148/) - HNer jedberg runs this

- ShareThis: 250 instances (I worked here)

Longest-running instance I've had is at two years and still going strong.

grep• 16 years ago

Is Reddit profitable?

jedberg3• 16 years ago

reddit currently has 73 instances with this breakdown:

  25 c1.xlarge
  26 m1.large
  22 m1.xlarge

c1.xl are generally app servers, m1.xl are generally databases (either postgres or cassandra) and the m1.l are other stuff.

jm33• 16 years ago

[i got permissions from Lachlan Donald at 99 Designs to post his email to me here - jm3]

We [at 99 Designs] actually don't have a huge number of instances, although we are steadily adding more and more.

We have at present:

  2 x Caching Proxies (Large)
  8 x App Servers (Large)
  3 x DB Servers (High-CPU X-Large)
  2 x Workers (Large)

We started out with a lot more small instances, but found that the disk performance on the small instances is extremely variable and often terrible.

We also use RightScale for management and are at present looking at using server arrays to scale up and down the App Server layer to deal with load spikes.

The biggest thing that reduced our need for lots of servers was moving as much functionality to asynchronous tasks as possible, which are queued on each appserver via Beanstalk queues and then consumed by the worker servers. Additionally we have as many pages http cache-able as we can, which are served directly from the squid cache at the front of the app.

Hope that helps!

Cheers, Lachlan Donald CTO, 99designs

imp• 16 years ago

I've got two running now. A large for the webserver and a medium high CPU for the database. Roughly a couple hundred reqests per second.

teej• 16 years ago

Zynga runs over 12,000 EC2 nodes total. A single app like FarmVille, which runs entirely on EC2, will be a few thousand. It's safe to guess we're Amazon's biggest customer.

mark_l_watson• 16 years ago

What size instances do you run for different types of functions?

jm33• 16 years ago

Does your number of instances scale up and down frequently, or does it monotonically increase with growth?

jedberg3• 16 years ago

You're not their biggest customer, but you're up there. :)

Ask HN: Max no. of EC2 instances used by a single web app?

Related Stories