News for hackers. Stuff that matters.
Re:Facebook's photo storage rewrite by thwarted on Monday April 06, 2009 @01:56AM (#548412)
Removed the reliance on expensive proprietary solutions, but how many non-standard custom tools needed to be written (and thus maintained) to verify, backup and access stuff? All that "extra" POSIX stuff could be avoided with a DOS formatted file system, or one of the many filesystems designed for small embedded storage systems (read and append only). Sites with the scalability and size of Facebook or Flickr ("...Facebook's effort to design a more cost-effective and high-performance storage system for their unique needs") may have a use for this, but the majority of smaller sites would most likely be best served by a direct filesystem store which has known semantics, easy access, and proven history.

I'd really like to see some performance numbers that shows which parts of their old storage system started breaking down at which levels of scale.

Re:Facebook's photo storage rewrite by Andys on Monday April 06, 2009 @07:14AM (#548674)
At the centre of this redesign is the need to save money.

Facebook doesn't need to run on the same hardware (NetApp) as banks and telcos. In this type of application, scaling up is done cheapest with custom software on commodity hardware.

Re:Facebook's photo storage rewrite by jrockway on Monday April 06, 2009 @10:37AM (#548897)
Additionally, the custom software is pretty straightforward. It's certainly simpler than the rest of their app.
Re:Facebook's photo storage rewrite by lyime on Monday April 06, 2009 @03:18AM (#548497)
If you go though the older presentation. It breaks down at the disk I/O level. Not only that, it seems like one of the main reason was cost. CDNs can get expensive, especially when are the biggest Photo service on the internet.
Re:Facebook's photo storage rewrite by thwarted on Monday April 06, 2009 @04:10AM (#548547)
I wasn't really including CDN in "storage system", even though it is listed in those graphics.
Re:Facebook's photo storage rewrite by chris on Monday April 06, 2009 @04:15AM (#548550)
More specifically, it broke down at metadata operations.