News for hackers. Stuff that matters.
Re:When It Comes to Facebook Scale, You Can Throw Out the Ru... by aristus on Wednesday September 24, 2014 @12:29PM (#8362383)
I used to be a server monkey long long ago, and played with the mockups of these servers that were set up at FB HQ. One thing to realize is they are heavy. Loaded with hard drives, one of these servers is roughly the size, shape and weight of a long 120mm mortar shell.

Easier to rack than the Dells of yesteryear, sure. But there can be up to 120 of these bad boys in a single standard rack, which means 60 of them have to be lifted above waist height during rack and stack. That is pretty strenuous, and technically you should use a moving lift and / or a lifting partner. Especially when you're talking about hundreds or thousands of racks.

Thinking outside the box and keeping the poor floor staff in mind, I've always wondered why we don't build little catwalks or half-height rack units, or even moving racks, like a dry cleaner's.

Re:When It Comes to Facebook Scale, You Can Throw Out the Ru... by delluminatus on Wednesday September 24, 2014 @01:28PM (#8362786)
This comment reminds me of my time as a sysadmin in college. I had to rack a lot of 1U and 2U servers (the Dells of yesteryear referred to) and they are EXTREMELY heavy. Racking them was tricky because you had to put together the rack mount yourself and then get it all lined up while you're holding this 100+ lb thing (which is awkwardly too long and wide to handle easily) at chest height and sliding it into the rails in the cabinet.

One day they let me take a decommissioned PowerEdge server back to my dorm room. It was old, but it still had something like 16 cores, 64 GB RAM, and 2 hard drives. As a college student with no money I thought this was pretty cool. Carrying that thing five blocks to my dorm room was one of the hardest damn things I've ever done, and in the end it was too loud to use.

Moral of the story: Rack-mount servers are not designed to coexist with humans.

Re:When It Comes to Facebook Scale, You Can Throw Out the Ru... by judgardner on Wednesday September 24, 2014 @02:16PM (#8363121)
Server lifts are pretty common kit in most colocations.

http://www.material-lifts.com/client_images/catalog19960/pag...

Re:When It Comes to Facebook Scale, You Can Throw Out the Ru... by peterwwillis on Wednesday September 24, 2014 @03:01PM (#8363431)
Barring those (and the space to fit it between the racks), you could attach tracks on the ceiling and use a moving chain & pulley to move them around with just one person. Granted, with double racks you'd still need a ladder to attach the chain.
Re:When It Comes to Facebook Scale, You Can Throw Out the Ru... by jedberg on Wednesday September 24, 2014 @01:23PM (#8362752)
I used to do rack and stack in the eBay datacenter. We actually had double height racks (84U). I always wondered the same -- wouldn't it be easier to build a catwalk instead of having a bunch of ladders and lifts around that required two people?
Re:When It Comes to Facebook Scale, You Can Throw Out the Ru... by aristus on Wednesday September 24, 2014 @01:33PM (#8362817)
Fourteen-foot-high racks? In earthquake country? I guess that's cheaper than a reinforced second floor + HVAC etc. But what the eff?
Re:When It Comes to Facebook Scale, You Can Throw Out the Ru... by jedberg on Wednesday September 24, 2014 @05:16PM (#8364199)
No biggie when they're bolted to the floor. :)

Also, cost of land trumps safety apparently.

Re:When It Comes to Facebook Scale, You Can Throw Out the Ru... by wmf on Wednesday September 24, 2014 @12:42PM (#8362459)
Do they rack it with the drives or do they rack it empty and then install the drives?
Re:When It Comes to Facebook Scale, You Can Throw Out the Ru... by aristus on Wednesday September 24, 2014 @12:50PM (#8362515)
With the drives. They really are little torpedoes.
Re:When It Comes to Facebook Scale, You Can Throw Out the Ru... by ControlledBurn on Wednesday September 24, 2014 @01:41PM (#8362869)
For the server nodes that have storage in the sled, that's true, but with the JBODs they generally rack the the sled first and then add each drive.
Re:When It Comes to Facebook Scale, You Can Throw Out the Ru... by ianamartin on Wednesday September 24, 2014 @12:45PM (#8362480)
I don't see anything in this article that sounds like throwing out the rulebook. It sounds a lot like throwing out a few stupid rules. But not the whole book.
Re:When It Comes to Facebook Scale, You Can Throw Out the Ru... by forgottenpass on Wednesday September 24, 2014 @02:49PM (#8363340)
Of course it sucks, It's techcrunch, they can't actually write about tech. A bunch of vaguely clued in non-techies writing about tech companies for AOL don't target an audience that wants to hear about server hardware. They want to hear about how great (innovative, rule-breaking, etc...) facebook is. If server hardware is the vehicle for that story, so be it.

This article is "company finds business value in customizing critical component instead of using COTS." And that's only interesting to people who understand the components. So they represent it poorly, but only a little bit, not badly enough to merit the weight we typically associate with the word "misrepresentation".

Re:When It Comes to Facebook Scale, You Can Throw Out the Ru... by hawkice on Wednesday September 24, 2014 @02:57PM (#8363400)
[Of course, but here is a little story]

I recently heard of a legal principle having to do with what they termed "overbroad" interpretation. The idea is, if there is a law in the United States that could be used to violate the constitution, the whole law gets struck down -- this is often limited to e.g. a subsection. But notably, it doesn't change the law to be the same except in cases where it violates the constitution, it just implodes the law. There's something to be said about the principle of throwing away any system or subsystem that doesn't get the job done, and not just forking and patching it.

Re:When It Comes to Facebook Scale, You Can Throw Out the Ru... by papercruncher on Wednesday September 24, 2014 @03:46PM (#8363671)
[unrelated] As someone who used to rack a lot of 120mm mortar shells, I empathize