News for hackers. Stuff that matters.
Re:Redis: under the hood by antirez on Tuesday March 08, 2011 @12:48PM (#2301804)
What changed since this article was written:

1) The Server structure in the unstable branch is finally divided into sections using comments.

2) The command table is now loaded into an hash table, so the lookup is now O(1) and it is possible to alter the hash table at runtime. This is how we implement ed the command renaming and shadowing in redis.conf.

3) The shared integers are not used when diskstore is enabled, since we need tha t every object is independent with a different LRU timestamp and so forth. But o therwise using the default mode of operations (in memory) shared integers are us ed.

4) Databases are now watched for the implementation of "WATCH" using a better AP I, that is signalModifiedKey().

5) Now we listen to the TCP socket and to Unix socket if configured. The Redis C luster branch (not public) will also listen to an additional TCP port using for cluster message passing.

7) Loading data from disk to memory is now non blocking, we do this re-entering the event loop while loading data.

8) Response reading is now faster and cleaner code, thanks to Pieter's work.

This is a very nice overview. Of course only the surface is covered, there are a lot more things going on in the specific commands, in VM / diskstore command (by far the most complex part of Redis), in specially encoded data types, and in rdb/AOF loading/saving, incrementally expanding hash tables, ...

In just 30k lines of code there is already a lot to take in mind!

Basically I and Pieter already reached the point where we are experts in different areas. We both understand the whole Redis codebase, but for instance he is more expert in the sorted sets and special encoding code, I'm more expert in the diskstore things, and so forth.

Re:Redis: under the hood by jeangenie on Tuesday March 08, 2011 @01:17PM (#2301928)
can anyone with some technical expertise comment on the quality of this write-up?
Re:Redis: under the hood by antirez on Tuesday March 08, 2011 @01:18PM (#2301932)
It's very good.
Re:Redis: under the hood by tomstuart on Tuesday March 08, 2011 @01:22PM (#2301957)
This might be my favourite HN exchange ever.
Re:Redis: under the hood by koenigdavidmj on Wednesday March 09, 2011 @12:43AM (#2303998)
http://news.ycombinator.com/item?id=35079

- Did you win the Putnam? If not, please don't be `bolder' than this guy? - Yes, I did.

Re:Redis: under the hood by mahmud on Wednesday March 09, 2011 @06:25AM (#2304577)
I told that one IRL, and people wanted to look up this "HN" thing.

Epic!

Re:Redis: under the hood by grammaton on Tuesday March 08, 2011 @05:30PM (#2302863)
Yhea, it's pretty solid.
Re:Redis: under the hood by pohl on Tuesday March 08, 2011 @02:29PM (#2302235)
antirez, did you delete a comment here? I see a dead one that appears to be very informative, and I'm wondering why it didn't live.
Re:Redis: under the hood by ehsanul on Tuesday March 08, 2011 @04:07PM (#2302597)
Deleted comments don't go dead, as far as I know. I think it was killed by someone by mistake (there are a few overzealous moderators here, or it may just have been a mis-click, or maybe even some algorithmic spam-killing, though I doubt that last one). Here's the comment from antirez:

    What changed since this article was written:
    1) The Server structure in the unstable branch is finally divided into sections using comments.
    2) The command table is now loaded into an hash table, so the lookup is now O(1) and it is possible to alter the hash table at runtime. This is how we implement ed the command renaming and shadowing in redis.conf.
    3) The shared integers are not used when diskstore is enabled, since we need tha t every object is independent with a different LRU timestamp and so forth. But o therwise using the default mode of operations (in memory) shared integers are us ed.
    4) Databases are now watched for the implementation of "WATCH" using a better AP I, that is signalModifiedKey().
    5) Now we listen to the TCP socket and to Unix socket if configured. The Redis C luster branch (not public) will also listen to an additional TCP port using for cluster message passing.
    7) Loading data from disk to memory is now non blocking, we do this re-entering the event loop while loading data.
    8) Response reading is now faster and cleaner code, thanks to Pieter's work.
    This is a very nice overview. Of course only the surface is covered, there are a lot more things going on in the specific commands, in VM / diskstore command (by far the most complex part of Redis), in specially encoded data types, and in rdb/AOF loading/saving, incrementally expanding hash tables, ...
    In just 30k lines of code there is already a lot to take in mind!
    Basically I and Pieter already reached the point where we are experts in different areas. We both understand the whole Redis codebase, but for instance he is more expert in the sorted sets and special encoding code, I'm more expert in the diskstore things, and so forth.
Re:Redis: under the hood by pg on Tuesday March 08, 2011 @05:00PM (#2302756)
It was a dupe (he probably hit submit twice). The dupe got autokilled. Then he noticed the duplication and deleted the original.
Re:Redis: under the hood by antirez on Tuesday March 08, 2011 @05:04PM (#2302769)
Exactly that. But I can see my comment in this page, I just killed one of the two exact copies.
Re:Redis: under the hood by pohl on Wednesday March 09, 2011 @10:18AM (#2305068)
Thank you for explaining. The other one didn't appear for me until after you answered. Odd.