Google Shows How To Scale Apps From Zero To One Million RPS, For $10

http://www.forbes.com/sites/reuvencohen/2013/11/26/google-shows-how-to-scale-apps-from-zero-to-one-million-requests-per-second-for-10/

by zafirk • 13 years ago

126 68 13 years ago

forbes.com

chuhnk2• 13 years ago

Having beta tested the Google Compute Engine while working at Google, I have only good things to say about it. They've come along way in an extremely short space of time and knowing what they have internally, this is just the beginning. They'll far surpass AWS for performance, ease of use and cost.

threeseed• 13 years ago

They'll far surpass AWS for performance, ease of use and cost.

Most likely. But those aren't the only things developers care about when looking for PAAS.

There is that whole "Platform" aspect. And AWS destroys Google in this respect. It has far more offerings and more importantly it has a very large ecosystem of companies who will be in the same data centre who you can leverage e.g. Iron.io.

toomuchtodo2• 13 years ago

Don't forget support. I know I can get decent AWS support if I pay Amazon. Google? HAH!

dudus• 13 years ago

Original post http://googlecloudplatform.blogspot.ca/2013/11/compute-engin...

wiremine• 13 years ago

The gist describing how to reproduce the test yourself: https://gist.github.com/voellm/1370e09f7f394e3be724

scoot3• 13 years ago

The only reason I can see for this being this high on the front-page [18 points and only one comment(from a googler)] is the google employee block vote (apparently tech-savvy isn't a prerequisite in the google upvote marketing department):

1 million requests per second is 20 times greater than the throughput in last year’s Eurovision Song Contest, which served 125 million users in Europe No, r/s != Gb/s.*

(Cue the down vote from the google up voters)

bgentry4• 13 years ago

Have you worked extensively with ELBs? They need pre-warming in advance of significant traffic changes, they can't handle spikes well because they take many minutes to respond to changes in traffic volume, and they use an ever-changing set of IP addresses for your load balancer nodes.

The GCE load balancer has none of these problems, which makes it a huge advantage over AWS and ELBs.

Disclaimer: I'm an engineer at Heroku. We manage dozens of ELBs for ourselves, and thousands of them for our customers.

ancarda• 13 years ago

They need pre-warming in advance of significant traffic changes

Why?

mattmanser• 13 years ago

Out of interest, what is a rough idea of the total requests per second on heroku for everything? All your nodes or whatever you call them? Dynamos?

Is 1,000,000 request per second a stupid, pointless number as no-one ever gets 1,000,000 requests per second or is this some sort of meaningful number?

tehwebguy• 13 years ago

I don't know man, someone's codepen with some pretty CSS is pretty high on the front page with 10 pts and 0 comments.

nl2• 13 years ago

Anything that can process 1MM req/sec gets upvotes from me.

1 million requests per second is 20 times greater than the throughput in last year’s Eurovision Song Contest, which served 125 million users in Europe No, r/s != Gb/s

Not sure what the point you think you are making is here, but the Eurovision site was tested to 50,000 req/sec[1]. 50000 * 20 = 1MM

[1] http://googlecloudplatform.blogspot.com.au/2013/05/how-scalr...

argonaut2• 13 years ago

I don't work at Google, but when people say something like "cue the downvotes" or "I know I'm going to get downvoted," I proceed to downvote them because that's an incredibly stupid and meaningless thing to say. In fact, it reeks of arrogant stubbornness ("I don't care about the community, I know I'm right, etc." Knowing you're right is not bad in of itself, it's being arrogant about it that is).

So I downvoted you.

scoot3• 13 years ago

Thanks, but I was specifically referring to the same voting ring that appeared to have put the article on the front page in the first place. I would hope the rest of the community would recognise a blogspam repost of a marketing puf-piece puff piece when they see one.

mayanksinghal• 13 years ago

I didn't upvote the submission, I didn't downvote you - but if you call people shills when they are not, some of them might. And I find 1M R/s impressive, but you can ignore it if you think it is trivial.

p3rfguy5• 13 years ago

Throughput is a measure of work over time. So here I was referring to throughput in terms requests per second and not Gb/s. So it was a direct comparison of 1M RPS (Compute Engine Load Balancing) vs 50K RPS (DNS Load Balancing).

Anthony F. Voellm Google Cloud Performance Engineering Manager @p3rfguy

krallin3• 13 years ago

To add a bit of context to Anthony's post here (I'm the guy who wrote the blog post on the Eurovision).

GCE Load Balancing uses a single IP address, you can point your DNS there and forget about it.

What Anthony's post shows is that this IP address will be able to serve 1 (upwards?) of 1 million requests per second.

This matters because you have control over scaling your backend (design properly, add more instances), whereas you don't have full control over scaling your frontend.

Indeed, a problem we had during Eurovision was that mobile providers (it was a mobile app) would cache the IPs of our frontend Nginx servers, so scaling those wasn't as easy.

So this new GCLB essentially "solves" scaling your frontend. That's something I'd care about ;)

Hope this helps shed some light here!

laz2• 13 years ago

DNS load balancing isn't all bad. Google uses it to host www.google.com for example ;)

drakaal2• 13 years ago

This is the reason that our site. http://plexisearch.com uses Google App Engine.

Google has allowed us to run "Code" rather than worry about infrastructure. Because it is an "Engine" we don't have to configure much, and the autoscaling is fantastic.

Google Edge Cache means we get Better than CDN performance on static assets.

There are limitation to AppEngine, but because we can also leverage virtual servers we can create a hybrid environments that let us do things AppEngine won't like install C Libraries, or run Windows (we aren't doing that, but we could).

We have been very happy with Google App Engine and since we are running millions of pages a day through our Natural Language Engine it has worked out really well for us.

-Brandon Wirtz CTO Stremor.com

bgentry4• 13 years ago

AFAICT this article is talking strictly about Google Compute Engine and its GCLB load balancer product, not about App Engine.

drakaal2• 13 years ago

How do you think App Engine works? AppEngine runs on the same hardware and the same process described as a how to is nearly identical to what is obfuscated and provided to AppEngine users via a GUI.

recuter• 13 years ago

I keep checking to see whether or not I can run my own images yet but Google Compute Engine so far supports less things than even Digital Ocean.

What if I don't want to run your version(s) of CentOS/Debian, guys? Hmm? (I hope somebody from there reads this) :)

Still, its a very very interesting offering and something to keep an eye on. I bet nobody here is using it yet because its still early days and the thread devolves into such old classics like "you'll never be able to reach a human at Google" and "they changed up their pricing with Google App Engine that one time and my app was no longer free to run all of a sudden!". Sigh.

I personally think it would be great if you could spin up the same instances on either cloud and load-balance/fail-over as needed/cheapest. Docker makes it doubly exciting.

bgoldy2• 13 years ago

Stay tuned - Getting custom image support and custom kernels so you can run things like Docker is just around the corner.

-Brian (@bgoldy)

p3rfguy5• 13 years ago

You can build, create, deploy and use your own images. Check out https://developers.google.com/compute/docs/images

Anthony F. Voellm Google Cloud Performance Engineering Manager @p3rfguy

yeukhon• 13 years ago

I am pretty excited and yet shocked to see what Compute Engine has evolved into. The original Compute Engine was about scaling (as cluster) in scientific experiment and yet this news makes me think Compute Engine is becoming the new App Engine but with full control of the VM (plus the amazing autoscaling feature). I always like to control my own VM because I can do much more with a VM than a sandbox (to me an App Engine is just a sandbox loaded with X framework and X database). I have always wanted to work on Compute Engine :(

This is an interesting marketing stragery. People who wish to launch a VM can choose CE and people who just want a sandbox quickly they can use App Engine? Though I am really skeptical about the future of App Engine if the CE became cheaper. I am sure if that happens, Google will do everything it can to migrate things over to CE. This is probably many years down the road...

I still think CE is really good for computations.

enneff2• 13 years ago

App Engine is not just a place to run your code. It has a nice deployment system with the ability to flip between versions instantly. It provides a few great pieces of core infrastructure: the datastore, task queues, and memcache (there are many others, but those are the essentials). The SDK gives you a stand-alone development server for testing locally. If you want to build web apps (or a backend for mobile apps), App Engine is a great way to go.

Compute Engine is for all the other stuff that you can't run on App Engine.

Incidentally, the datastore is now available as a stand-alone service, Google Cloud Datastore: https://developers.google.com/datastore/ This should benefit Compute Engine users.

hrjet• 13 years ago

I like App-Engine and now that there are open-source, API compatible alternatives, there is no lock-in to worry about either.

The biggest drawback for App Engine is lack of async support. The only ways to scale are: multiple-threads (slow) or multiple instances (costly).

rbanffy• 13 years ago

The main difference, for me, is that with EC2 (and GCE), my team manages the servers. With App Engine, it's Google's Site Reliability Engineers who do it.

thrownaway2424• 13 years ago

Compute Engine is not for scientific experiments, or at least not specifically. Perhaps you are thinking of Exacycle? http://research.google.com/university/exacycle_program.html

bad_user3• 13 years ago

I'm not sure about Google's CE, I don't really know what it can do, but Amazon's AWS has had for some time auto-scaling capabilities by means of auto-scaling groups coupled with their ELB load-balancer. We used it and it worked out great.

danhoc01• 13 years ago

$10 for a 10 minute test, okay. A month of 200 US-hosted n1-standard-1 machines would cost 200 * $.115 * 30 (days) * 24 (hours)... over $16,000. That doesn't include load-balancing, bandwidth, or any additional charges there may be. The price shouldn't be mentioned anywhere in the article.

Demand is elastic of course, and if you really want to scale in a cost-effective manner you also need to do auto-scaling. As far as I can tell (I have no direct experience with GAE), it's much easier on AWS. It would also be interesting to see if you can scale your pool of webservers from 1 -> 200 faster on AWS or GCE.

The article does quote @cloudpundit, who hits on the true point of the exercise: Relevance of GCE LB load test: With AWS ELB, if you expect a big spike load, must contact AWS support to have ELB pre-warmed to handle load. I would also guess that Amazon is working to improve ELB to behave similarly, especially now that Google's product has less restrictions than theirs.

espeed• 13 years ago

GCE instances are charged in 1 min increments (https://cloud.google.com/pricing/compute-engine), not 1 hour increments like AWS (http://aws.amazon.com/ec2/pricing/).

1 min increment billing enables you to spin up massive Web clusters to handle spikes or Hadoop clusters so big that you can process the entire dataset it a few minutes, while paying less than you would for a smaller AWS cluster that's billed by the hour.

_delirium• 13 years ago

It's definitely finer-grained, but there's still a 10-minute minimum per instance: billing is only per-minute above 10 minutes. So if you run a massively parallel 2-minute Hadoop job, you pay num_instances x 10min. Admittedly that's still a win, since AWS would bill you for 6x as much instance time. But it's not a 30x differential as you might guess from the headline "hourly" vs. "per-minute" pricing.

tlawrence• 13 years ago

The point here isn't necessarily about the actual elasticity of the instances themselves, but the load balancer in front of them. With ELBs, they can only handle so much traffic so quickly. You literally cannot just point a million requests per second at it, even if you have 200 instances supporting it, and handle them all right off the bat. The ELBs themselves scale with the load, and currently, that scaling is slow during spikes.

rartichoke• 13 years ago

I don't think $16,000 a month is really that bad if you factor in what type of money a consistent 1 million reqs/sec would probably give you.

1m req/s is:

60 million+ reqs an hour

1.4 billion+ reqs a day

43 billion+ reqs a month

It would be a weird situation to be serving that much traffic but not being able to afford hosting.

It would be cool if someone could estimate the cost of what it would take to sustain 1m reqs/s with dedicated hardware or maybe an unmanaged VPS cluster.

bhauer• 13 years ago

I only skimmed the gist with the instructions for reproducing the test [1], but in that skimming I could not figure out what the request and response were in this simulation. Were these responses 100 bytes, 1k, 10k, 100k? I guess small responses, but I'm not certain.

I know the goal of the test is to demonstrate how capacity can be scaled upward in very little time and with comparatively little effort. And for that this demonstration is impressive!

That said, the requests per second number is not especially impressive given that a modest single server can easily saturate gigabit Ethernet with 100-byte responses (and it's even easier to do so with larger responses) [2]. I am left wondering, does the $10 cost cover the cost of server instances and bandwidth? If so, that is a very good deal. The bandwidth charges for exceeding the capacity of a gigabit Ethernet connection (1M RPS with even the most trivial requests requires more than 1 gigabit) would be substantial with many hosting providers.

[1] https://gist.github.com/voellm/1370e09f7f394e3be724

[2] http://www.techempower.com/benchmarks/#section=data-r7&hw=i7...

scoot3• 13 years ago

Couldn't agree more. As I (poorly) tried to point out in my earlier off-the-cuff comment, requests-per-second is a meaningless measure without the context of the size of those requests. The Forbes repost and the original google marketing article are both vague on details, and appear to conflate requests-per-second with throughput.

p3rfguy5• 13 years ago

Marketing Post.. Ouch :) One of the things I love about working at Google is we in engineering are empowered to share our knowledge. For those wanting the details check out the step-by-step instructions in the Gist (https://gist.github.com/voellm/1370e09f7f394e3be724).

The goal was to measure the speed of scaling and load balancing vs egress. Bigger egress would not change the load balancing decisions.

Anthony F. Voellm Google Cloud Performance Engineering Manager @p3rfguy

jsnell• 13 years ago

The blog post says 1 byte responses excluding HTTP headers. Since it was a standard Apache configuration, the headers would probably be something like 200-300 bytes.

p3rfguy5• 13 years ago

Thats about right. In hindsight a tcpdump or other trace might have been useful to include.

Anthony F. Voellm Google Cloud Performance Engineering Manager @p3rfguy

rallison3• 13 years ago

Indeed. On a singular i7 machine with a gigabit connection, a number of frameworks are in the 400,000-600,000rps range when returning super trivial responses. In comparison, from the original Google blog post:

To demonstrate scaling of the Compute Engine Load Balancing fanout we used 200 n1-standard-1’s Web Server running Apache v2.2.22 on Debian 7.1 Wheezy Images. Users are encouraged to use larger VM types for better single machine backend web serving, however here we demonstrated the scaling of the load balancer to backends and were not concerned with the backends themselves using every cycle to serve responses. Each backend web server received ~5K requests per second, which is an even distribution.

So, to match the peak rps of solid (but not top of the line) dedicated hardware appears to take upwards of ~120 instances of n1-standard-1 (assuming that it scales linearly, of course). Not a trivial number.

That said, I am impressed at how quickly this can scale up. If you have a site that normally runs fine on a couple of instances, but occasionally sees massive spikes in traffic, this could make sense. And from a purely engineering point of view, GCE and EC2 are quite interesting.

p3rfguy5• 13 years ago

We really wanted to focus on the scale of operations for this article so we used a full http request / response. The payload of the response was 1 byte.

Anthony F. Voellm Google Cloud Performance Engineering Manager @p3rfguy

jheriko• 13 years ago

you do need context... a million things a second is not a great metric. these are small numbers compared to a lot of things. e.g. how many pixels your graphics card is pushing right now.

nl2• 13 years ago

BTW, the Gartner analyst (Lydia Leong[1]) who's tweet was quoted is worth following. She tells it like it is, with no bullshit (read some of her stuff about the various OpenStack debacles for example).

[1] https://twitter.com/cloudpundit

tzury• 13 years ago

This is the original post http://googlecloudplatform.blogspot.ca/2013/11/compute-engin...

judk• 13 years ago

RPS and $ are not interconvertible units, unless that 1M RPS is financed by the interest on a $10 investment.

bad_user3• 13 years ago

AWS has a really interesting feature in their auto-scaling groups, coupled with their ELB (elastic load balancer). You can configure an auto-scaling group with a variable number of instances that ads or removes instances automatically, based on the ELB's measured latency or the number of requests coming in or many other such metrics.

This works out great and results in cost savings, as it can survive spikes, plus during the night the traffic is at most half or even less than the traffic you get during the day. I had a setup that was handling over 30,000 reqs/sec and during the night it kept about 6 h1.medium instances active, while during the day it could go upward to 20 instances, but was usually stable at around 14 instances.

This article mentions ELB, but I don't understand - does Google's Cloud Compute offer something similar? Can one vary the number of instances based on the incoming traffic or other metrics?

krallin3• 13 years ago

Google Compute added an ELB equivalent (the "GCLB"), but it doesn't have autoscaling built-in.

To get autoscaling, you can either roll your own — Google explains how here[0] —, or use a multicloud autoscaling solution (disclaimer: I work for such a provider).

[0] https://cloud.google.com/resources/articles/auto-scaling-on-...

bad_user3• 13 years ago

Thanks for the link. I wonder how well it works. Did you try implementing it?

adrr• 13 years ago

Google's load balancer offering doesn't need to prewarmed was the main point. Its a fair point, having to prewarn ELBs isn't exactly fun, form takes about 10 minutes to fill out. If you're doing TV ads, lots of stuff is last minute. You can prewarm it yourself by throwing artificial traffic at it but you always risk just taking down the ELB if you ramp up to fast. Weird things happen to ELBs if you spike them with big bursts of traffic, one of service providers had 24 hour backlog of web hooks that they sent out all at once. It caused our ELB to fail all the instances even if we brought up new ones. Also during load you can get other sites to show up on your ELB. Had asp.net error page show up on ours. We run rails. Wish amazon would scale up ELBs instances based on the number instances you have behind it using the ratio calculated on past usage.

X4• 13 years ago

The cake is a lie, they don't teach howto scale yourself. They teach howto use THEIR infrastructure to scale. I mean it's still really useful to know and all, but the headline makes the impression that Google teaches you howto scale your own company's App from Zero to 1Million RPS.

tobyn• 13 years ago

This is misleading, $10 is just for the duration of the test. Roughly $58k/mo if $1.33/min.

laz2• 13 years ago

The big thing they're crowing about is because you can spin something like this up quickly on demand in the G cloud.

Most sites aren't remotely close to this artificial traffic pattern (1 packet request, 1 packet response).

It's kinda cool from an L4 load balancing perspective that it's only one fault tolerant IP address. In terms of L4 LB throughput though, a single box with IPVS will happily do 1M pps.

Google Shows How To Scale Apps From Zero To One Million RPS, For $10

Related Stories