Redis: under the hood

http://pauladamsmith.com/articles/redis-under-the-hood.html

by cannedprimates • 15 years ago

85 12 15 years ago

antirez3• 15 years ago

What changed since this article was written:

1) The Server structure in the unstable branch is finally divided into sections using comments.

2) The command table is now loaded into an hash table, so the lookup is now O(1) and it is possible to alter the hash table at runtime. This is how we implement ed the command renaming and shadowing in redis.conf.

3) The shared integers are not used when diskstore is enabled, since we need tha t every object is independent with a different LRU timestamp and so forth. But o therwise using the default mode of operations (in memory) shared integers are us ed.

4) Databases are now watched for the implementation of "WATCH" using a better AP I, that is signalModifiedKey().

5) Now we listen to the TCP socket and to Unix socket if configured. The Redis C luster branch (not public) will also listen to an additional TCP port using for cluster message passing.

7) Loading data from disk to memory is now non blocking, we do this re-entering the event loop while loading data.

8) Response reading is now faster and cleaner code, thanks to Pieter's work.

This is a very nice overview. Of course only the surface is covered, there are a lot more things going on in the specific commands, in VM / diskstore command (by far the most complex part of Redis), in specially encoded data types, and in rdb/AOF loading/saving, incrementally expanding hash tables, ...

In just 30k lines of code there is already a lot to take in mind!

Basically I and Pieter already reached the point where we are experts in different areas. We both understand the whole Redis codebase, but for instance he is more expert in the sorted sets and special encoding code, I'm more expert in the diskstore things, and so forth.

jeangenie• 15 years ago

can anyone with some technical expertise comment on the quality of this write-up?

antirez3• 15 years ago

It's very good.

tomstuart• 15 years ago

This might be my favourite HN exchange ever.

grammaton• 15 years ago

Yhea, it's pretty solid.

pohl2• 15 years ago

antirez, did you delete a comment here? I see a dead one that appears to be very informative, and I'm wondering why it didn't live.

ehsanul• 15 years ago

Deleted comments don't go dead, as far as I know. I think it was killed by someone by mistake (there are a few overzealous moderators here, or it may just have been a mis-click, or maybe even some algorithmic spam-killing, though I doubt that last one). Here's the comment from antirez:

    What changed since this article was written:
    1) The Server structure in the unstable branch is finally divided into sections using comments.
    2) The command table is now loaded into an hash table, so the lookup is now O(1) and it is possible to alter the hash table at runtime. This is how we implement ed the command renaming and shadowing in redis.conf.
    3) The shared integers are not used when diskstore is enabled, since we need tha t every object is independent with a different LRU timestamp and so forth. But o therwise using the default mode of operations (in memory) shared integers are us ed.
    4) Databases are now watched for the implementation of "WATCH" using a better AP I, that is signalModifiedKey().
    5) Now we listen to the TCP socket and to Unix socket if configured. The Redis C luster branch (not public) will also listen to an additional TCP port using for cluster message passing.
    7) Loading data from disk to memory is now non blocking, we do this re-entering the event loop while loading data.
    8) Response reading is now faster and cleaner code, thanks to Pieter's work.
    This is a very nice overview. Of course only the surface is covered, there are a lot more things going on in the specific commands, in VM / diskstore command (by far the most complex part of Redis), in specially encoded data types, and in rdb/AOF loading/saving, incrementally expanding hash tables, ...
    In just 30k lines of code there is already a lot to take in mind!
    Basically I and Pieter already reached the point where we are experts in different areas. We both understand the whole Redis codebase, but for instance he is more expert in the sorted sets and special encoding code, I'm more expert in the diskstore things, and so forth.

pg• 15 years ago

It was a dupe (he probably hit submit twice). The dupe got autokilled. Then he noticed the duplication and deleted the original.

antirez3• 15 years ago

Exactly that. But I can see my comment in this page, I just killed one of the two exact copies.

Redis: under the hood

Related Stories