News for hackers. Stuff that matters.
Re:Staircar: Redis-powered notifications by bmatheny on Wednesday July 20, 2011 @05:32AM (#2784844)
I worked on Staircar so maybe I can answer a couple of questions.

Machine failures: failures we'll likely address in a future post but basically each redis instance can have a slave which can be failed over to.

Performance: Staircar isn't nearly as fast as redis, but the primary project goal was making the redis infrastructure opaque to any clients. The secondary goal was performance, which is still quite good.

Early Optimization: In a large infrastructure (thousands of machines, multiple data centers, etc) you have locality issues, slow clients, machine failures and other operational considerations. Again, redis performance is fine and we weren't concerned with that. We were interested in creating a high performance proxy to a pool of redis instances. Also, with Staircar we can make online changes to the redis pool without needing to notify clients of those changes.

Re:Staircar: Redis-powered notifications by SteveMorin on Wednesday July 20, 2011 @03:48PM (#2787123)
Is the code open source or do you have plans to open source it. I am actually pretty interested in this and would love to be able to read the code for it. -Steve
Re:Staircar: Redis-powered notifications by bmatheny on Thursday July 21, 2011 @08:55AM (#2789276)
Hey Steve, we do have plans to open source the code in two parts. The first part to likely be open sourced will be the high performance logging library that was written for it. After that, the proxy component. Keep an eye on the Tumblr engineering blog for a notification.
Re:Staircar: Redis-powered notifications by SteveMorin on Thursday July 21, 2011 @11:35AM (#2789926)
Thanks
Re:Staircar: Redis-powered notifications by a-priori on Wednesday July 20, 2011 @08:14AM (#2785130)
I find it interesting that there are two distinct bands in the response time plots. Any ideas what the two represent?
Re:Staircar: Redis-powered notifications by bmatheny on Wednesday July 20, 2011 @08:36AM (#2785171)
The upper band essentially represents new users. At the current rate of growth, we push into a new redis shard every 2-3 days. When we first start writing/reading against that shard, it appears to take a bit of time to warm up.
Re:Staircar: Redis-powered notifications by rdoherty on Tuesday July 19, 2011 @08:07PM (#2783681)
Neat technology, but at first I thought I was going to read an article about some sort of messaging/event system. It's not; Staircar is basically a way to deal with high write to read ratio data. It's not for subscribing to or publishing events.
Re:Staircar: Redis-powered notifications by mkjones on Tuesday July 19, 2011 @09:27PM (#2783894)
This looks pretty cool. I'm curious how they deal with machine failure though - does Staircar coordinate some kind of redundancy, or when you lose a redis machine do you need to manually fix it and bring it back up with whatever the last checkpoint data was?
Re:Staircar: Redis-powered notifications by troyk on Tuesday July 19, 2011 @11:36PM (#2784191)
I'm curious as to the speed difference between redis current protocol and a resty-json loving http redis server, as the speed was more than enough for tumblr and going the http route has the benefit that any http client becomes a full blown redis client.
Re:Staircar: Redis-powered notifications by aaronbrethorst on Tuesday July 19, 2011 @11:31PM (#2784181)
Do you have to watch out for hop-ons?

edit: no arrested development fans, i see...

Re:Staircar: Redis-powered notifications by stephth on Wednesday July 20, 2011 @12:53AM (#2784355)
It is a pretty good naming scheme.

Looking forward for steveholt, featherbottom, annyong and nevernude.

Re:Staircar: Redis-powered notifications by jrockway on Wednesday July 20, 2011 @01:30AM (#2784445)
There's always money in thebananastand.
Re:Staircar: Redis-powered notifications by aaronbrethorst on Wednesday July 20, 2011 @01:54AM (#2784493)
Steveholt!

Also: MrF, Cornballer, and Chickendance.

Re:Staircar: Redis-powered notifications by ozataman on Tuesday July 19, 2011 @09:08PM (#2783853)
It reads like a hash-based sharding front-end for Redis rather than a layer for notification systems specifically.
Re:Staircar: Redis-powered notifications by owyn on Wednesday July 20, 2011 @01:58AM (#2784505)
Possibly early optimization syndrome? Redis is awesome, but the main problem here seems to be the amount of data (in the hundreds of gigs) and not scaling writes... The stated requirements are 7500 notifications per second but a single instance of redis can do 50k/sec updates on my mac...