Nginx and Memcached, an easy 400% boost in req/s

http://www.igvita.com/2008/02/11/nginx-and-memcached-a-400-boost/

by igrigorik • 18 years ago

22 13 18 years ago

Many dynamic sites can't just cache entire pages and bypass the appserver completely. It would require a lot of changes to the application code.

igrigorik2• 18 years ago

Yup, definitely true. Having said that, we tend to overuse this thinking even when we don't really have to. This is arguably a scheme that's better adapted to new codebases. :)

wmf• 18 years ago

It's amazing to me that caching still isn't on by default in Web servers.

eusman• 18 years ago

what kind of caching? why would you want a web server cache that would eat up memory? what you need is an independent distributed dumb cache in memcached terms.

ivankirigin• 18 years ago

I'm going to blog it in detail soon, but I thought I'd mention that tipjoy's front page is entirely static. The content is refreshed very often, and a new file is saved. Right now lighttpd is setup to serve the static page when / is requested. Soon, it will be nginx because of the ease of using it as a load balancer.

disconnect2• 18 years ago

Sloppy thinking. Sloppy measurements.

If you have almost 1000 reqs/sec initially, that's 1ms per request. By bringing this down by the amazing 400% (!!) percent you've shaved off 3/4th of a milisecond per request. Congratulations - you have no idea what you've been measuring. Just the way in which the request is logged can make a 5ms difference. Insignificant and completely uninteresting.

Back when 3d game programming was cool, people would get a rotating cube and show off their 1200 fps frame rate. Of course they would get upset that by adding a texture to the cube the framerate dropped by 400 frames to 800 fps. "Textures are slow!", they'd say. Yeah, right...

igrigorik2• 18 years ago

Fair enough, but this is / was a controlled experiment. The codebase did not change, the method through which the cached content was served did. Hence the textured cube argument is not the best analogy here, I think. :)

Having said that, I agree -- there is a lot of other variables out there, and they should not be swept under the rug.

disconnect2• 18 years ago

The point of the textured cube analogy was that if you don't consider the context of the benchmark, the numbers aren't useful.

The kid doesn't realize that if texturing a cube brings a framerate down from 1200 to 800 fps, is as significant as going down from 62 fps down to 60. Even though that is only a 2 fps difference.

A 400% performance gain is a constant performance improvement. I don't think a system like nginx + memcached have linear performance benefits in the first place.

tx2• 18 years ago

I refuse to believe in memcached. Apache, along with other web servers, has a built-in in-memory cache for static content. Decent SQL-servers have several caches on different levels, in fact those caches in theory should be far more effective than a simple remote hashtable that memcached is. And all that - on top of OS level disk I/O caching.

I suspect that memcached was born as a result of sloppy SQL querying multiplied by inefficient SQL schema and lack of RDBMS tuning skills. Moreover, anyone can have memcached with zero programming by either hosting DB on a RAM disk with replication, or using in-memory tables.

jbert• 18 years ago

My hunch is that it's down to cache invalidation logic. It's easier for the app to know when a single memcached entry needs to be invalidated than for the SQL engine, which perhaps tends to be conservative in what it can cache.

(e.g. MySQL query cache is flushed if a single write happens to the table, so all your users will be dumped out of that cache whenever any row is updated.)

Hence you get a better hit-rate with memcached. Perhaps it is also simply easier to tune memcached than most db memory caches.

I benched "retrieve data and instantiate object" on a couple of different, unloaded, systems and a simple select-and-instantiate-from-mysql was a bit faster than the memcached equivalent. I didn't measure CPU usage server-side though, so under load things might be different.

tx2• 18 years ago

I agree, in-memory hash table controlled manually by the application has a million ways to beat any SQL-server caching.

What I am saying is that I suspect that memcached is way overused by people who don't know much about SQL. I am not posing like a DB snob, in fact my skills are fairly modest, but even I know that query caching is just one of the many caches SQL servers employ, and well designed servers easily service thousand+ of requests per second. And that's just one instnace.

Very very very few web sites ever reach such loads. In theory memcached should be more like a very niche product for such rare sites, yet it appears that everybody is using it.

I am against it because it increases the complexity. Memcached deployment and cache control become two more things to get right.

That's all.

brianr• 18 years ago

The next time you're designing the backend for a high-traffic site, I think you should give it another look. As jbert pointed out, the MySQL query cache is emptied every time you make any change to the table, making it pretty useless unless you only ever do selects. And even for the selects, wouldn't you rather select by key from a hashtable in O(1) than select from a btree index in O(log n) at best?

nostrademons• 18 years ago

Brad Fitzpatrick addressed that when he released the initial memcached - LiveJournal had spent about 2-3 years fiddling with MySQL caches to get decent performance. They even took out one of those MySQL support contracts and had a MySQL engineer in to troubleshoot their performance problems.

The bottom line was that yes, DBs should cache things - but they don't. The main issue was as someone else mentioned, the query cache is flushed whenever the table's written to, which makes it essentially useless on a multi-user website.

Also, memcached distributes the cache itself across multiple machines, which makes a huge difference to cache hit rates. When you have a local query or app cache in a multi-server setup, you end up with each cache storing the same data, so the size of the cache is limited by the amount of RAM you have in the machine. If you distribute the whole cache, you're limited by the amount of RAM in all machines, which means almost every request can hit the cache. Hence you have architectures like FaceBook's 3 TB memcached setup.

Nginx and Memcached, an easy 400% boost in req/s

Related Stories