Backblaze      $117,000
  [...]
  Amazon       $2,806,000

Is backblaze's solution cheaper than S3? Absolutely. But they're also twisting the numbers a bit.

99.99% uptime means roughly 1 hour per year of downtime. I don't know what the specific failure rates on the components are, but it seems reasonable that A) the data drives are hot-swappable, and will not cause downtime when they are replaced and B) the rest of the components fail once a year (or less) and take ~10 minutes to replace and reboot the system. With 4 main places of failure (PSU, Boot Drive, Motherboard/Ram/CPU, Drive controllers), as long as you have staff constantly on call and they can respond within 5 minutes of a failure, 99.99% uptime seems reasonable.

I don't know where you got the idea that your data was on 4 different servers when using S3. I can't find even the slightest amount of information on that. Yes, that would be nice, and it is cool to think that, but its rather doubtful that they're actually doing that (or they could probably add another 9 to their uptime).

"Amazon keeps at least 3 copies of your data (which is what you need for high reliability) in at least 2 different geographical locations. "

I can't find the original source supporting that statement, but I also know it to be true based on direct contact with AWS team. (I'm using AWS since private alpha of EC2 in 2005)

In addition, looking at the actual S3 contract, they're really only guaranteeing at 99.9% uptime, which allows for up to 8 hours of downtime a year, more than enough to completely re-build the outer server once a year, as long as they can keep the data intact (which they seem more than capable of with their setup, once again assuming their data center is not completely destroyed).

I believe that S3 replicates to multiple physical locations. So, while you might experience downtime, you probably won't experience data loss.

That is just the cost to buy all the components they have. If you budgeted that in you'd just have a bunch of boxes at your office that weren't even wired up. On the other hand, from Dell or Sun you at least have all the hardware in the chassis, if not some basic configuration and OS install done for you. Go with Amazon and you've already got it racked up in a data center with dedicated techs and replacement parts on hand.

It isn't a fair comparison.

Tell you want, since you're a fellow HNer, I'll save you some money and do it for half that. I'll even throw in some basic configuration. You're welcome!

It might not add up the same for your needs (assuming you want multi-data center redundancy of all data, etc...).

They're going Google's route, which is fine, it's just... a whole different direction than a Sun Thumper. I'd be interested to see how their costs per petabyte stack against Google's.

Google's operation has at most half the disks-per-rack density of Backblaze, but with 4 or 8 times the server density. Google does almost all their computation on the same nodes where the data is stored. Google is also storing each tablet a minimum of 3 times within each cluster, with most systems having multiple replica clusters.

Backblaze's system is going to have several orders of magnitude less bandwidth:

  way too many drives in one server (some of them on the PCI bus!)
  use of port multipliers (causing 5 drives to share one SATA cable's 300MB/s)
  RAID6 with too many drives per array (15-way XOR is no fun)
  why use JFS?
  only access is via HTTPS, not clear if SSL is done on the pod

The more painful corollary is that, even if one measures not by total storage but by performance (a more critical aspect of, e.g., databases), the prices are still an order of magnitude or two apart.

I would argue that the reason "No One Sells Cheap Storage" in the backup/archive sense is that there isn't enough demand. Obviously, this is starting to change.

I am, however, startled that a driveless pod is so expensive ($2467 or $54.82 per drive), considering that external performance is, effectively, limited by gigabit Ethernet.

Two SuperMicro 846E1s would be $2k for 48 drive bays, and one could easily connect 10 of them to a single $300 SAS RAID card. Add another $1k for mobo and cables, and that's $11300 for 240 drives or $47.08 per drive, without having to do a custom case.

Granted, it takes up almost twice the rack space, but it's still a decent power density at 50W/U AC (assuming 6.5W per drive DC).

It takes 52 groups of 13 1.5TB disks (Hmm.. 13 pops up an awful lot in their setup) to make a PB using their pods. 51 groups means 17 pods is .9945TB for $134k ($134.5/PB).

Using my pods, it would be .936TB for $120.3k ($128.5/PB), but what's $6k/PB between friends?

I don't think a single SAS card can support more than 128 (or maybe 256 if you're lucky) disks. This doesn't affect your math much, but I wouldn't want someone to try this at home and be disappointed. If you're not willing to build a custom case and don't feel like suffering the vagaries of SATA expanders, LSI SAS cards plus Supermicro JBODs are the way to go.

Similarly, from "at home" experience, I can also safely say that luck factors prominently in the vagaries of SATA port multipliers. The reason I even made the attempt was a lower price per port. This is why I'm so started that a solution based on them would end up being more expensive than a SAS-based one.

Of course, if the camel of performance gets its nose into the tent of ones requirements, it has to be the SAS way or the highway.

Personally, I'm working on a system whereby I just fill up otherwise unused slots on my chassis and use that for backup. The system isn't done yet, but I've got 6 disks out there waiting for me to figure it out.

If I only needed one I'd seriously consider it, as my time, for setting up one unit and working out the kinks, would unquestionably be more than that, and they have quite a bit of experience working out the problems by now, even if I made my system redundant, their system would unquestionably be more reliable the first iteration.

However, I plan on needing a bunch more, so it probably makes sense for me to roll my own, work out the kinks, and hopefully end up with something cheaper that has a lot more cache.