News for hackers. Stuff that matters.
Re:So, that was a bummer by rbranson on Tuesday October 05, 2010 @10:33PM (#1762478)
MongoDB is designed to help ease the pain of scaling, but what you're asking for is magic. If all the sudden a large amount of requests start coming through that overload a shard (think very popular users, like if several very popular users ended up colocated on a shard), how is MongoDB going to anticipate this? Any kind of shard scaling will only work well in scenarios where you have a reasonable increase in load, not a crippling surge.

In addition, it's very, VERY difficult to scale writes against a single object, such as Justin Bieber's profile data, say, if you've got a view counter on it. You can either serialize writes on read like Cassandra does, which has it's own drawbacks (the more writers an object has, the more expensive reads become), or you can have single-master-for-an-object sharding like MongoDB employs and most other production sites (Facebook, Flickr, etc) use.

Re:So, that was a bummer by fauigerzigerk on Wednesday October 06, 2010 @04:44AM (#1763200)
How is this primarily an issue of overlaod? One particular shard got overlaoded, yes, but the reall issue is how that brought the whole system down.
Re:So, that was a bummer by Devilboy on Tuesday October 05, 2010 @10:43PM (#1762494)
There was no sudden crippling surge in this case as far as I can tell. There was no mass-updating of a single object either. It failed even though it fits your ideal situation pretty much perfectly.