How Google Taught Me to Cache and Cash-In

http://highscalability.com/how-google-taught-me-cache-and-cash

by timf • 17 years ago

58 24 17 years ago

desu4• 17 years ago

Seems to mainly be about Reddit.

Didn't seem to answer the #1 question I had in my mind - how do they cache votes for logged-in users? Do they, say, cache a generic front page and then apply the user's prior voting choice via JS? Or do they cache the page only for the general public, and for logged-in users just cache their voting history and generate it on-the-fly?

However they do it, they do a damn good job. I'm always impressed by how responsive Reddit is, along with the other highly dynamic / high load sites like Digg, etc. They manage to stay up and running fine even with all that going on; but just a link on their front page to someone's blog, which should be almost static, crushes that server into goo. Shows the power of good design!

joe_the_user• 17 years ago

This probably isn't how they do it but you could have a static page updates customizing information with AJAX.

bmelton2• 17 years ago

In the past my strategy was basically that.

Load a static page from cache wherever possible. If the user performs a vote, or submit, or what have you, write it to the database, but just perform the update on the user's local page in the DOM.

desu4• 17 years ago

Yeah, that's what you'd do as the user makes a vote. But how would you load the page in the first place? The front page is different for each user, as they've likely voted up and down numerous links on it.

You could load a generic page and then apply their votes by AJAX or something but that doesn't strike me as being any more efficient for the DB than just generating the page.

uggedal• 17 years ago

Use the source Luke: http://code.reddit.com/

desu4• 17 years ago

Thanks. I want to know the answer, but not so much that I'm willing to spend the several hours necessary to familiarise myself with a large foreign code base. I was hoping someone here could give a quick top-level explanation.

From looking around, though, I think they're caching the user's votes and then just assembling the page on the fly from cache fragments. You could do that pretty quickly. Might be wrong.

jules• 17 years ago

You could load the user's votes into a cookie.

desu4• 17 years ago

It would get too large quickly.

Check this out for a good explanation of why you want to keep cookie sizes as small as possible: http://yuiblog.com/blog/2007/03/01/performance-research-part...

alexandros2• 17 years ago

I wonder if anyone's gone the next step and written updates to the caches directly, leaving out the database write from the critical path.

DrewHintz• 17 years ago

And then what happens when the server with your cache dies? :)

axod• 17 years ago

Update the cache instantly (critical path), update the db lazily queued?

laut• 17 years ago

Do you mean that when you get an update, you both write to the database and perform an update in the cache? If so, you would want it to be a transaction so that the cache and database is not out of sync.

Maybe a write cache, like on hard drives with write back caching. If you first wrote the update to a cache (queue of updates). Then it would be written to both the cache (cached html for instance) and database in one transaction.

But I think it sounds like more trouble than it's worth.

alexandros2• 17 years ago

transactions are unnecessary, unless you are looking for 100% consistency between cache and database at all times. Usually this is not the case and if it is, you will have extremely serious problems scaling. All i'm saying is: If you have an update that you know will invalidate some cached pages, why invalidate them and then rerender them from the database once the update is commited instead of updating them directly. and lazy queueing the update, taking the load of the database (less pressure to commit instantly, and less reads). In the facebook inbox example, you know that a new message to the user will increase their message count by one. Just update the cache directly with this new info. When/if it explodes, by all means, rerender everything.

jasonkester2• 17 years ago

We do this for the Twiddla sandboxes.

The key is that you have to be prepared for everything to evaporate if the server cycles its memory for whatever reason. In this specific case, we toss everything out of the sandboxes every 5 minutes & reset them anyway, so it's no great tragedy if they clean up after themselves from time to time.

For something with a ton of low-priority edits (like HN or Reddit where nobody sues you if their vote goes away), you could certainly get away with caching updates and only saving them out every once in a while.

Edinburger• 17 years ago

Another idea: populate ('warm') the cache automatically after server restarts to get up to speed quickly.

amix2• 17 years ago

Following this guide blindly is a bad idea. Doing premature caching can be dangerous - you may end up caching stuff that gets hit once, you may miss caching stuff that get hit a lot of times and you may end up making your caching a lot more complex than it needs to be.

A better caching strategy is to collect data (for example, what are the most used queries) and then do caching optimization based upon this data. I.e. don't apply caching blindly, but apply it by profiling your application. This approach also works for testing out how new caching strategies affect your cache and your database.

lucifer• 17 years ago

You let the cache expire stale objects and update it on db IO. What's the problem, exactly?

amix2• 17 years ago

Updating the cache is far from easy. An example that show cases the non-triviallity is MySQL's query cache: it will purge the whole cache on a table update. If you get a lot of reads, this works great, but if you get a lot of updates, then MySQL query cache will be very inefficient as the cache will be purged on every update. MySQL query cache has a lot information available that could be used implement a smarter cache, unfortunately this problem is far from easy to solve even with a lot of information available.

The bottom line: good luck on expiring the cache on database updates ;)

ecq• 17 years ago

For people familiar with Oracle, 11g has caching built-in called result cache. it's transparent and works well.We saw a 400-800+% increase in performance in our application by just turning this feature on, without any change in our application.Also note that you can still use memcached on top of result cache for even better performance/scalability.

jasonkester2• 17 years ago

One extra point: Don't Add Caching Yet.

1. wait until something presents itself as a bottleneck

2. optimize it until it's not a bottleneck anymore

3. wait until it presents itself as a bottleneck again anyway.

...Then add caching.

You'll never find the low-hanging yet dog-slow fruit if you put your aggressive caching scheme in place right off the bat. If you have a bunch of poorly optimized code running with a ton of caching to hide it from you and you suddenly have enough traffic to cause scaling pain, that's a big problem.

Speed it up. Then cache it.

How Google Taught Me to Cache and Cash-In

Related Stories