News for hackers. Stuff that matters.

A comment on Simple Twitter clone using only the Redis key-value store as db and PHP. Parent comment

Re:Simple Twitter clone using only the Redis key-value store... by antirez on Thursday May 21, 2009 @08:57AM (#620337)
Absolutely true. This is why Redis can do things that with an on-disk storage engine are almost impossible. This our reasoning: in order to really scale, with every kind of DB, if data access is evenly distributed like happens in many web 2.0 apps, the DB must fit in memory anyway (being this OS disk cache, or directly used memory like in Redis). What really matters is persistence and replication. Asynchronous disk dumps gives us all this: data is persistent, replication is fast and non blocking and trivial to setup. And because everything is in RAM we are free to do high level stuff without to care about how data is stored on disk. On disk we have just a "stupid" dump that we reload a startup.

In some way this is what Paul Graham described in some comment years ago: take all the data in memory in your application, and just write a file in append-mode to reload it later. Basically this is an ad hoc, informally-specified, bug-ridden, slow implementation of half of Redis :)

[ Parent | on HN ]
Re:Simple Twitter clone using only the Redis key-value store... by petercooper on Thursday May 21, 2009 @05:02PM (#621128)
On disk we have just a "stupid" dump that we reload a startup.

In a lazy fashion, or not? That is, if I have a 10GB data set, will I be either waiting X minutes for a basic restart or dealing with sluggish queries for a slightly lesser time?

(Update for anyone who wants an answer too: I dug around the docs and found that it's a full read into memory on restart, not lazy loading. They give an example of "It takes about 45 seconds to restore a 2 GB database on a fairly standard system, no RAID." So if frequent restarts are in your plan for some reason - not that they should be, of course - it might not be a goer.)

[ Parent | on HN ]
Re:Simple Twitter clone using only the Redis key-value store... by spoiledtechie on Thursday May 21, 2009 @09:29AM (#620392)
Ya but what happens when Solid state hard drives come into play. And they will be soon.

I, a asp.net guy will be able to compete with Redis then.

[ Parent | on HN ]
Re:Simple Twitter clone using only the Redis key-value store... by antirez on Thursday May 21, 2009 @10:09AM (#620453)
If RAM gets so cheap that Solid state hard drivers will come into play you'll have your Linux box with 512 GB of Ram by default. Btw I'm happy with ASP guys competing with Redis :) Also if someone stole what's good about Redis and creates FooBaredDB that is just better I'll drop Redis and use FooBaredDB.
[ Parent | on HN ]
Re:Simple Twitter clone using only the Redis key-value store... by pierrefar on Thursday May 21, 2009 @11:04AM (#620541)
Interesting. Thanks for the detailed explanation!

I'll be checking it out in detail (hopefully) soon.

[ Parent | on HN ]