News for hackers. Stuff that matters.
Re:Sun engineer responds to the Backblaze "Petabytes on a bu... by jacquesm on Thursday September 03, 2009 @03:10PM (#803234)
That's a pretty weird analysis, comparing a system that is special purpose with a general purpose one.

He's concentrating on disk-to-board throughput, but this being network attached storage on a LAN, using the motherboards' single ethernet port the amount of data read or written to those drives per second will never exceed 1 Gbit/second anyway.

I agree with the reliability analysis though, backblaze seems to be on fairly thin ice there.

Still, it seems to work for backblaze, I'm taking it as read that they've lived through a bunch of drive failures by now and they seem to still be in business.

Re:Sun engineer responds to the Backblaze "Petabytes on a bu... by sp332 on Thursday September 03, 2009 @03:25PM (#803286)
"That's a pretty weird analysis, comparing a system that is special purpose with a general purpose one."

I think that's why he says right at the beginning:

  This device is that cheap because it cuts
  several corners. That's okay for them. But
  for general purpose this creates problems.
  I want to share my concerns just to show you,
  that you can't compare this to a X4540 device.
Re:Sun engineer responds to the Backblaze "Petabytes on a bu... by jacquesm on Thursday September 03, 2009 @03:54PM (#803369)
And then he goes and does the comparison anyway...
Re:Sun engineer responds to the Backblaze "Petabytes on a bu... by akamaka on Thursday September 03, 2009 @04:06PM (#803399)
Let's not forget that BackBlaze started this, by making a graph that compared their prices to S3, etc. (and were also criticised for that in the previous HN thread).
Re:Sun engineer responds to the Backblaze "Petabytes on a bu... by Retric on Thursday September 03, 2009 @04:51PM (#803519)
They are talking about cloud storage. Having 3 live copy's of your data in 3 locations is far more important than having lot's of redundancy at a single location. If you want redundancy just add more locations. Large scale company's like Google, Microsoft, and Yahoo understand this but sun is selling feel good hardware to companies that like to think they have unusual needs.

As to speed, nothing you do is going to reduce a disk's low latency so talking about bandwidth is mostly a waste of time. Device bandwidth is rarely the limitation once you start to scale, because you can add devices to your network faster than you can build a better network.

Re:Sun engineer responds to the Backblaze "Petabytes on a bu... by sp332 on Thursday September 03, 2009 @04:04PM (#803395)
No, he contrasts them :)
Re:Sun engineer responds to the Backblaze "Petabytes on a bu... by uibuibk on Thursday September 03, 2009 @05:35PM (#803622)
Ironic though how Sun spent the last 20years trying to persuade companies that networks of Unix mini-computers were a cheap reliable alternative to mainframes - yet when it comes to putting disks in boxes you need to have everything stamped with the magic 'enterprise' badge.

If you believed their spin presumably you would buy from a proper enterprise company like IBM or EDC - not some failed workstation vendor.

Re:Sun engineer responds to the Backblaze "Petabytes on a bu... by rythie on Thursday September 03, 2009 @07:32PM (#803837)
If you look at SANs in general, Sun undercuts that whole space, especially if you look at what they do with the 7000 series which combine SSDs and HDDs.

http://www.sun.com/storage/disk_systems/unified_storage/

Re:Sun engineer responds to the Backblaze "Petabytes on a bu... by russss on Thursday September 03, 2009 @07:44PM (#803859)
The 7000 series is an incredibly cool system, and it's definitely giving NetApp/EMC a run for their money.

That said, it's still really expensive if you're willing to go that extra mile and design your own system for your needs. And it's really not that hard.

Re:Sun engineer responds to the Backblaze "Petabytes on a bu... by rythie on Thursday September 03, 2009 @07:59PM (#803891)
Yeah I totally agree it's expensive - though compared to the other enterprise storage solutions it's cheap. That said people should be making cheaper solutions when you don't that enterprise level system - especially if you can handle the failures in your software stack.
Re:Sun engineer responds to the Backblaze "Petabytes on a bu... by sh1mmer on Thursday September 03, 2009 @03:29PM (#803299)
I think the can of worms is that Backblaze themselves compared their system to more general solutions. While they were doing that for their own purpose they weren't so explicit with that point that many people (such as GigaOM, etc) took it as a general comparison.
Re:Sun engineer responds to the Backblaze "Petabytes on a bu... by jacquesm on Thursday September 03, 2009 @03:59PM (#803383)
They explicitly said that it was a storage unit/$ comparison, not throughput, not reliability.
Re:Sun engineer responds to the Backblaze "Petabytes on a bu... by Andys on Thursday September 03, 2009 @08:35PM (#803971)
It would have helped if they'd published graphs next to it showing the same list of systems sorted by throughput and another sorted by reliability!
Re:Sun engineer responds to the Backblaze "Petabytes on a bu... by jacquesm on Thursday September 03, 2009 @10:59PM (#804190)
For sure. I think one of the big problems with the original article is that the fabriacator of the project did the comparison and failed to see the suggestion that was being made. Probably not on purpose, but plenty of folks ran with it thinking they could have all of 'cheap, fast, reliable'.

They just got 'cheap', reliable comes at the expense of using multiple of these units and fast looks like it is going to be impossible without striping across multiple setups like this.

Interesting for an extremely narrow use case (long tail storage, think backup services, video sites, photo sites).

Outside of that maybe useable with some adaptation but not 'as is', besides that it is not as though their machines are up for sale, the figures they quote are mostly after all the r&d is already factored out, if you had to do this for a one-off, just cutting and folding that case would cost a pretty penny.

I'm fairly sure there is a market for a device that is a more robust version of this.

Probably at a 50% or so premium compared to the one they showed you could get a wider spectrum of uses and a bit more robust construction.

I would have bought one three months ago, now I had to settle for 20T or so in one 4U box, for a price a little over half of what they quoted in the article.

Re:Sun engineer responds to the Backblaze "Petabytes on a bu... by onedognight on Thursday September 03, 2009 @04:00PM (#803388)
Backblaze is making a backup server. They don't need to handle power supply failures. If one goes down, you'll replace it long before someone needs their backup. They don't need to worry about disks failing while writing (zfs and battery backup). Hell they can restart from scratch any one backup that encounters disk failures. They have all the time they want to do a backup.
Re:Sun engineer responds to the Backblaze "Petabytes on a bu... by dlsspy on Thursday September 03, 2009 @07:01PM (#803788)
> you'll replace it long before someone needs their backup

How do you know when I'll need my backup?

Re:Sun engineer responds to the Backblaze "Petabytes on a bu... by jacquesm on Thursday September 03, 2009 @07:23PM (#803825)
Because you can access it from that other server that bb kept, and maybe even those two other servers if they're halfway smart.

Replication is what it is all about.

Re:Sun engineer responds to the Backblaze "Petabytes on a bu... by ntoshev on Friday September 04, 2009 @01:20AM (#804319)
Statistically it would be a rare case for a customer to request a backup restore from a failed unit even if you don't replicate. Bb could handle these with polite mails explaining your data will be available in 24 hours and offering you some free service as compensation - it wouldn't hurt them, they do consumer backups.
Re:Sun engineer responds to the Backblaze "Petabytes on a bu... by buugs on Thursday September 03, 2009 @03:31PM (#803306)
He did a good job explaining why one should shop around and weigh their options... Such that it works for backblaze but that doesn't mean it will work on other applications especially concerning power failure and vibrations. He did a very good job of objecting to there message professionally regarding its use for others.
Re:Sun engineer responds to the Backblaze "Petabytes on a bu... by anigbrowl on Thursday September 03, 2009 @03:37PM (#803325)
I think his 'DC-3' analogy is valid - BB has a decent system if you don't mind the risks, Sun is selling peace of mind.
Re:Sun engineer responds to the Backblaze "Petabytes on a bu... by uibuibk on Thursday September 03, 2009 @05:25PM (#803596)
>Sun is selling peace of mind

Right upto the point where they are sold to a database company that isn't interested in selling disks.

the nice thing about the blaze is that if seagate decide to double their prices or much you to a new expensive platform - you just buy toshiba instead.

Re:Sun engineer responds to the Backblaze "Petabytes on a bu... by Hoff on Thursday September 03, 2009 @04:36PM (#803484)
FUD and RAS being two of the basic tenets of marketing most anything in the enterprise IT space, after all.
Re:Sun engineer responds to the Backblaze "Petabytes on a bu... by rbanffy on Thursday September 03, 2009 @05:01PM (#803545)
There is a lot of good engineering in the Thumper. There is a lot of clever engineering in the Backblaze. The comparison states a lot of valid points on power distribution, drive reliability and RAID throughput.

I see Backblaze will learn a lot from building their storage systems and future versions may be much better, but this one cuts too many corners for me to feel confident it will have the same kind of reliability I would recommend.

What amazes me most is the use of desktop-grade hard drives. Not because they are sloppily built, but because their performance requirements are so different from a server environment.

This makes for some nice reading: http://cacm.acm.org/magazines/2009/6/28493-hard-disk-drives-...

Re:Sun engineer responds to the Backblaze "Petabytes on a bu... by sho on Friday September 04, 2009 @01:23AM (#804324)
Ask Joyent about the "peace of mind" they got from their Thumper ...
Re:Sun engineer responds to the Backblaze "Petabytes on a bu... by duskwuff on Saturday September 05, 2009 @12:46AM (#806207)
Re:Sun engineer responds to the Backblaze "Petabytes on a bu... by anigbrowl on Thursday September 03, 2009 @04:20PM (#803439)
Can anyone estimate the practical benefits (or not) of adding some extra Gbit network cards in the free motherboard slots on the BB device?
Re:Sun engineer responds to the Backblaze "Petabytes on a bu... by jsz0 on Thursday September 03, 2009 @05:57PM (#803656)
Probably not much. The disk IO of this system is probably pretty bad. It might not be able to saturate a single GigE connection. Additionally the data is going mostly to consumer residential internet users. It's doubtful they need more than one GigE connection per box.
Re:Sun engineer responds to the Backblaze "Petabytes on a bu... by lsc on Thursday September 03, 2009 @04:42PM (#803498)
You would have to do work (either 'etherchannel' or similar bonding, or load balance requests over 2 IPs) but in theory, if you had a high-bandwidth bus like pci-express, you should be able to get more bandwidth out of a chassis by adding more ethernet cards.

Me, I would add more cache at the same time. SATA tends to be really good for sequential, but be really, really bad for random. multiple sequential streams hitting the same disk start to look an awful lot like random access after a while.

Re:Sun engineer responds to the Backblaze "Petabytes on a bu... by jacquesm on Thursday September 03, 2009 @05:17PM (#803574)
They are using a consumer grade motherboard with one ethernet port, 3 PCI slots and one PCI express slot, the slots are all used up by the sata interfaces.
[deleted]
Re:Sun engineer responds to the Backblaze "Petabytes on a bu... by jacquesm on Thursday September 03, 2009 @03:17PM (#803261)
> Given the reliability info he cites, the fact that backblaze is still in business must mean their usage-to-storage ratio is small.

No, it probably means that they get their reliability at a different level, such as simply storing the data on a number of machines. Systems like glusterfs make this completely transparant.

Re:Sun engineer responds to the Backblaze "Petabytes on a bu... by abyssknight on Thursday September 03, 2009 @03:38PM (#803327)
Gluster is broken. Hadoop is pretty proven, but I haven't heard of single, non-academic Gluster deployment that worked. From personal experience I can honestly say that software cost us several customers, and caused a support nightmare for a good month before we bought a SAN instead.
Re:Sun engineer responds to the Backblaze "Petabytes on a bu... by jacquesm on Thursday September 03, 2009 @04:03PM (#803391)
> Gluster is broken

Agreed, but I do know a few situations where it is being used, the 'stack' has been undressed to the minimum and tested to the hilt before deployment.

Still, I would not risk my company on it. I've been a big fan of their architecture from pretty much day 1, but so far they seem to be feature oriented and not stability oriented.

Re:Sun engineer responds to the Backblaze "Petabytes on a bu... by bbgm on Thursday September 03, 2009 @07:42PM (#803853)
At least a couple of pharma companies actively use Gluster. Haven't used it myself, but have heard some stories (not positive) about the stability
Re:Sun engineer responds to the Backblaze "Petabytes on a bu... by notaddicted on Thursday September 03, 2009 @03:39PM (#803329)
( ahhh I deleted my post before I saw the response )

From the numbers he cited I got the impression that the drives they are using would fail 3+ times more often than enterprise drives (lower MTTF + speced at low temps). The cost of replacing the drives (even if they are 1/2 the price) would make the system uneconomical.

Also: because of their case design (non-hot-swappable) the cost of human replacement is higher too.

Thus they must have a low usage rate to have a low failure rate for their system to make business sense.

Re:Sun engineer responds to the Backblaze "Petabytes on a bu... by jacquesm on Thursday September 03, 2009 @03:53PM (#803367)
I think their use case is sufficiently a-typical to get away with this.

They're primarily write-only, only when a customer retrieves their data does stuff get read.

If they spin down the drives when the volume is not being accessed the mttf for a single drive goes up a lot, possibly beyond the point where it matters.

Re:Sun engineer responds to the Backblaze "Petabytes on a bu... by mattmcknight on Thursday September 03, 2009 @05:58PM (#803659)
That made me think of MAID (and COPAN), which is decent approach.

http://www.copansystems.com/

Re:Sun engineer responds to the Backblaze "Petabytes on a bu... by jrg on Friday September 04, 2009 @06:09AM (#804571)
Floor loading (and rack size) is a problem with their 'very big box of disks'.

That, and the fact that the company seems to be struggling to get sales.

Re:Sun engineer responds to the Backblaze "Petabytes on a bu... by rbanffy on Thursday September 03, 2009 @06:23PM (#803705)
Given proper redundancy and a adequately read/write unbalance, it's safe to power-down drives during read-heavy cycles and only power them up when you have some redundant data to write.

But you will have to manage the cycles for redundancy, lifetime and power savings. The software to do it must be really clever.

Re:Sun engineer responds to the Backblaze "Petabytes on a bu... by duskwuff on Saturday September 05, 2009 @12:49AM (#806214)
Or even to power down the entire pod if the data on it is quiescent. If you managed the fill and aging properly, this wouldn't even be hard to do -- as a backup service, they're write-heavy, which gives them a lot of flexibility that other applications don't have.