The C10K problem

http://www.kegel.com/c10k.html

by luu • 12 years ago

112 54 12 years ago

Last time I built a server project from scratch it was on a 300MHz quad-core Xeon base, which hosted around 150 simultaneous web users. It took some effort to make it scale like that but it was worth it. My hardware maintenance was low because we really maximized the software capacity.

In modern times, RAM and CPUs are more than 10 times bigger and faster, but I am seeing people get around 25 times LESS out of them, because they choose terrible tools, don't benchmark, and generally don't care. Now (different company) our app server pool has 150 nodes and each serves 4 users. The application complexity is significantly smaller than what I did 13 years ago.

I sincerely doubt any of my coworkers have read this document. It shows.

fiatmoney• 12 years ago

"It took some effort to make it scale"

It's often a lot easier to rely on horizontal scaling, especially on a strategic level when you're deciding whether to trust your organization's technical ability (which really means management's hiring ability) or your ability to call AWS and spin up another hundred servers.

"Easier" doesn't mean "cheaper" or "faster" in the long run, but it is lower-variance for sure.

philwelch• 12 years ago

Horizontal scaling isn't just about throwing hardware at the problem. It's about distributing your application over n number of nodes without single points of failure.

davidw• 12 years ago

A lot of the calculation is the $$$ you make per connection. For a SaaS kind of thing, you can generally keep that high enough to not worry about squeezing every last drop of efficiency out of things. A much bigger worry for most startup-ish companies is finding product market fit.

DrJ• 12 years ago

While I would agree that trying to squeeze every last drop of efficiency may not be worth the time/money for the company. I think the parent is implying that people are wasting a lot of extra resources due to poor tools, poor coding, poor profiling, poor maintenance, etc.

I would agree though that for startups, the problem is finding the place to $$$, it is important to know what the cost per user/connection/instance is and being able to control or monitor it.

gberger• 12 years ago

(1999)?

wging• 12 years ago

Arguably (2009) at the earliest. Was certainly originally written earlier but it seems like something of a living document. The first sentence after the table of contents is:

"See Nick Black's execellent Fast UNIX Servers page for a circa-2009 look at the situation."

jleader• 12 years ago

vangale• 12 years ago

I was definitely using this page as a resource in 2000-2001. I was going to try and use Python Medusa module (precursor of Twisted) to scale connections.

Aloisius• 12 years ago

Mmm. It has been updated over the years, but of course, 10K was relatively easy even back then. For fun one day, I turned off the search index on one of the production machines and hit over 90K at Napster without much trouble (production ran at ~36K). And that was on little dual processor Pentium 2 machines.

lugg3• 12 years ago

If you dont have a blog I think you should blog some stories from those days. I'd be really interested to hear them.

mturmon• 12 years ago

Yes, here is the original 1999 snapshot. It has been updated a bunch over the years.

http://web.archive.org/web/19990508164301/http://www.kegel.c...

I think Dan wrote this page originally when he was at Disappearing, Inc. Later he was at Google (2004-10).

dontdieych2• 12 years ago

If you decide to publish long serious article to web, please consider use css for better readability. Don't rely on browser default css too much. They are very busy to make javascript fast. Especially, please don't use 100% width. Sure, our displays have far more width than height. But that width is good for movie, not text. Put any books on your monitor. Then reduce text line width.

https://www.readability.com/articles/zrgovuxr

TW;DR: Too Wide, Didn't Read.

blago• 12 years ago

There is a bookmarklet for this: javascript:(function(){ document.body.style.width='600px'; })()

dontdieych2• 12 years ago

Mostly I'm fine with default css but only 100% width. This could be helpful in many old article like this. Thanks

ricardobeat• 12 years ago

This article is 15 years old. There wasn't much in the way of CSS back then, much less responsive design. But the brilliant thing is it still works perfectly, because it's so simple! Pop your tab to a window and resize, or use a readability browser extension/website. Oh, you just did that.

elwell• 12 years ago

haha yeah

Joeboy• 12 years ago

I can pretty much always read sites that don't bother to override the browser defaults, whereas people's CSS efforts often make things hard to read.

It's obviously possible to make sites nicer using CSS, but if you're just trying to let people read your words rather than show off your design skills, leaving it to the browser's defaults is absolutely fine.

lugg3• 12 years ago

If more sites did this maybe we'd get something a little nicer. Til then, you can always set your own defaults.

lugg3• 12 years ago

Just quietly this article has been around since 2003.. Its still an extremely useful resource, and FYI, chrome on android renders this page just fine. To further on that sentiment, coming from someone posting on hacker news I find your comment amusingly ironic.

pyalot210• 12 years ago

The 10k problem is still not solved. A run of the mill VPS typically manages around 1k simultaneous connections and 10k requests/s. That is, over plain http. Over TLS this goes down to about 100 connections and 600 requests/s. A cheap amazon instance will usually be around 1/5th of all that.

With a bit of tweaking you can get the plain http case a bit higher, but the TLS route will not get much better, because TLS isn't written with speed in mind (the handshake is slow and expensive) and the prevalent implementation (OpenSSL) is not written for high performance servers (it basically dictates that you run blocking sockets in a thread per connection).

Unfortunately spdy and various http2 proposals rely on TLS (in order to punch trough proxies), which means going back in server performance about 10 years.

So it is of little surprise that companies have started offering "cloud" solutions, because the typical VPS can't handle todays high traffic internet over TLS (worse than plain http by a factor of 10) and the typical cloud server is worse than a VPS by a factor 5, creating a 50x performance degradation, artificially. Obviously when faced with the question of running 10 servers, or 100, most small companies turn to the even worse "cloud" solution, requiring even more servers (500).

The whole affair is a sodding mess, and we're wasting massive amounts of energy and capital on insisting on doing things inefficiently. This is because by rights our VPS servers should easily be able to break trough the 10k limit in every way, but it can't because the OS wastes a lot of time running an inefficient network stack as well as that TLS and OpenSSL can't be bothered to get their act together.

And that is how, in the year of our lord 2014, more than 15 years after somebody writing the 10k problem article, and after webservers becoming at least 128x faster then back in the 90ties, most websites out there can still not stand up to serious traffic, take ages to load, and are hosted by an infrastructure (routers and whatnot) that easily buckle under even light DDOSing.

wbsun• 12 years ago

When it goes to transactions and stateful sessions, C10K, or even C1K is far from solved yet.

pyalot210• 12 years ago

But that's a bit absurd isn't it? A recent core i7 reaches 124850 MIPS. That means that if it takes 1/1000th of a second to handle a connection, the CPU could execute 124 million instructions during that time. At 10/10000th it could still execute 12 million instructions. A minimal asynchronous connection handling certainly doesn't require more than say 10'000 instructions, so our servers are under-delivering on their performance at least by a factor of 1200x, perhaps even by a factor of 12000x.

Our servers should be able to surpass C1k easily, even C10k shouldn't tax them. They should, by rights, only be taxed by the C10m problem.

Nursie• 12 years ago

There are many commercial alternatives to OpenSSL. I've worked with one or two. There's bound to be one that suits your needs. Of course you may also need to go to a full commercial, non-FOSS stack, but I'm sure plenty of companies will have those to sell too.

theknown997• 12 years ago

Not really true unless you use crappy software.

I run a few thousand simultaneous connections on AWS small instances without any issues. (HTTP + HTTPS).

The key is writing good software. Which a lot of programmers are still really bad at.

pyalot210• 12 years ago

You know, it's cute when somebody who wasn't even born when I started writing software for servers tries to discredit me.

seunosewa• 12 years ago

If you care about efficiency you should not use VPSs. You should dedicated servers.

snaky• 12 years ago

2013 update:

The C10M problem

It's time for servers to handle 10 million simultaneous connections, don't you think? After all, computers now have 1000 times the memory as 15 years ago when the first started handling 10 thousand connections.

Today (2013), $1200 will buy you a computer with 8 cores, 64 gigabytes of RAM, 10-gbps Ethernet, and a solid state drive. Such systems should be able to handle: - 10 million concurrent connections - 10 gigabits/second - 10 million packets/second - 10 microsecond latency - 10 microsecond jitter - 1 million connections/second

http://c10m.robertgraham.com/p/manifesto.html

peterwwillis• 12 years ago

Read the "Other Performance Metric Relationships" part at the bottom of this page[1]. Basically, just because your machine may be able to physically hold 10 million connections open, does not mean your machine could handle opening 10 million connections in a reasonable amount of time, much less handling 10 million transactions in a reasonable amount of time. If you can't open that many connections at once, or process that many transactions, just being able to keep them open becomes moot.

This article[2] breaks down the issues fairly well. In order to handle this kind of traffic, you have to basically redesign huge swaths of technology that exist because we don't want to have to implement these things more than once. I don't see how anyone would invest in this without a specific itch to be scratched (like deep packet inspection).

[1] http://www.cisco.com/web/about/security/intelligence/network...

[2] http://highscalability.com/blog/2013/5/13/the-secret-to-10-m...

jrmenon• 12 years ago

Big fan of kqueue() mentioned in the article. IIRC with sockets, it not only tells you if the socket fd is ready (say for non-blocking read), but even informs the number of bytes available to read which allows you write efficient code (i.e. not to some fixed buffer where you may need to loop in again to see if more data needs to be read).

Also I think for files/directories, you can listen for any changes that occur.

Wish this was officially available in Linux.

The C10K problem

Related Stories