Re:So, that was a bummer
by fauigerzigerk on Wednesday October 06, 2010 @04:44AM
(#1763200)
How is this primarily an issue of overlaod? One particular shard got overlaoded, yes, but the reall issue is how that brought the whole system down.
There was no sudden crippling surge in this case as far as I can tell. There was no mass-updating of a single object either. It failed even though it fits your ideal situation pretty much perfectly.
In addition, it's very, VERY difficult to scale writes against a single object, such as Justin Bieber's profile data, say, if you've got a view counter on it. You can either serialize writes on read like Cassandra does, which has it's own drawbacks (the more writers an object has, the more expensive reads become), or you can have single-master-for-an-object sharding like MongoDB employs and most other production sites (Facebook, Flickr, etc) use.