News for hackers. Stuff that matters.
Re:Petabytes on a budget: How to build cheap cloud storage by mmt on Tuesday September 01, 2009 @04:06PM (#798780)
As a sysadmin for organizations of different sizes, I've been acutely aware of this for quite some time.

The more painful corollary is that, even if one measures not by total storage but by performance (a more critical aspect of, e.g., databases), the prices are still an order of magnitude or two apart.

I would argue that the reason "No One Sells Cheap Storage" in the backup/archive sense is that there isn't enough demand. Obviously, this is starting to change.

I am, however, startled that a driveless pod is so expensive ($2467 or $54.82 per drive), considering that external performance is, effectively, limited by gigabit Ethernet.

Two SuperMicro 846E1s would be $2k for 48 drive bays, and one could easily connect 10 of them to a single $300 SAS RAID card. Add another $1k for mobo and cables, and that's $11300 for 240 drives or $47.08 per drive, without having to do a custom case.

Granted, it takes up almost twice the rack space, but it's still a decent power density at 50W/U AC (assuming 6.5W per drive DC).

It takes 52 groups of 13 1.5TB disks (Hmm.. 13 pops up an awful lot in their setup) to make a PB using their pods. 51 groups means 17 pods is .9945TB for $134k ($134.5/PB).

Using my pods, it would be .936TB for $120.3k ($128.5/PB), but what's $6k/PB between friends?

Re:Petabytes on a budget: How to build cheap cloud storage by wmf on Tuesday September 01, 2009 @06:46PM (#799095)
Two SuperMicro 846E1s would be $2k for 48 drive bays, and one could easily connect 10 of them to a single $300 SAS RAID card.

I don't think a single SAS card can support more than 128 (or maybe 256 if you're lucky) disks. This doesn't affect your math much, but I wouldn't want someone to try this at home and be disappointed. If you're not willing to build a custom case and don't feel like suffering the vagaries of SATA expanders, LSI SAS cards plus Supermicro JBODs are the way to go.

Re:Petabytes on a budget: How to build cheap cloud storage by mmt on Wednesday September 02, 2009 @08:05PM (#801434)
From experience, I can safely say that luck has nothing to do with it. LSI MegaSASes support more than 240, so it doesn't affect my math at all.

Similarly, from "at home" experience, I can also safely say that luck factors prominently in the vagaries of SATA port multipliers. The reason I even made the attempt was a lower price per port. This is why I'm so started that a solution based on them would end up being more expensive than a SAS-based one.

Of course, if the camel of performance gets its nose into the tent of ones requirements, it has to be the SAS way or the highway.

Re:Petabytes on a budget: How to build cheap cloud storage by lsc on Tuesday September 01, 2009 @06:45PM (#799094)
you might also want to check out coraid. it's at a similar price point. http://www.coraid.com/PRODUCTS/SR2421 - now, I've heard mixed experiences with it, and have no personal experience, but if you are talking about wiring up something yourself, it's definitely something to consider.

Personally, I'm working on a system whereby I just fill up otherwise unused slots on my chassis and use that for backup. The system isn't done yet, but I've got 6 disks out there waiting for me to figure it out.

Re:Petabytes on a budget: How to build cheap cloud storage by mmt on Wednesday September 02, 2009 @07:58PM (#801420)
Hmm.. looks suspiciously like exactly the SuperMicro case, for about twice the price of rolling ones own.
Re:Petabytes on a budget: How to build cheap cloud storage by lsc on Wednesday September 02, 2009 @08:37PM (#801479)
from what I understand the coraid systems are plan9-based, and only have a gigabyte of cache ram.

If I only needed one I'd seriously consider it, as my time, for setting up one unit and working out the kinks, would unquestionably be more than that, and they have quite a bit of experience working out the problems by now, even if I made my system redundant, their system would unquestionably be more reliable the first iteration.

However, I plan on needing a bunch more, so it probably makes sense for me to roll my own, work out the kinks, and hopefully end up with something cheaper that has a lot more cache.