News for hackers. Stuff that matters.
Re:StackOverflow Update: 560M Pageviews a Month, 25 Servers by bradyd on Monday July 21, 2014 @01:31PM (#8064812)
I don't believe that much RAM is uncommon for large scale database servers. 384GB RAM is only about $5000 from Dell. They also have a new server model coming out that supports up to 6TB of RAM [0].

[0] http://www.dell.com/us/business/p/poweredge-r920/pd

Re:StackOverflow Update: 560M Pageviews a Month, 25 Servers by valarauca1 on Monday July 21, 2014 @03:51PM (#8065635)
Most people over estimate hardware cost. Buying a 300-500GB ram server maybe very expensive. But buying a distributed data base expert likely costs at least an order of magnitude higher per year.
Re:StackOverflow Update: 560M Pageviews a Month, 25 Servers by jaequery on Monday July 21, 2014 @10:52PM (#8067200)
Same server in AWS, it'll cost $50k+/month for the same thing. SO just bought it one time fee for probably around $30k or so, I'll say they are doing something right!
Re:StackOverflow Update: 560M Pageviews a Month, 25 Servers by kondro on Monday July 21, 2014 @11:35PM (#8067279)
The don't have servers with that much RAM, but a 244GB RAM server with 8x800GB SSDs and 32 cores costs <$5k a month ad-hoc and about $1300/month with a 3 year commitment.
Re:StackOverflow Update: 560M Pageviews a Month, 25 Servers by valarauca1 on Tuesday July 22, 2014 @09:53AM (#8068963)
$1300/month w/ a 3 year commitment is $46.8k over the lifespan. Which is more then 4x more then the actual hardware cost. Your ROI on a physical server with the same stats at that price tag would be 8 months.
Re:StackOverflow Update: 560M Pageviews a Month, 25 Servers by habosa on Tuesday July 22, 2014 @02:07PM (#8070574)
Not saying its totally worth it, but the upside is that they take care of power costs, physical space costs, networking, and if the hardware fails they'll put your application on a new server. Basically you take all of the risk out the equation.
Re:StackOverflow Update: 560M Pageviews a Month, 25 Servers by egwor on Tuesday July 22, 2014 @03:21PM (#8071056)
Sure, but add power and cooling for these things plus physical space? (I remember when these big db's needed the floors in the data centre reinforcing since they were a > metric ton )

Still, I agree re. the difference in price. I'd buy too

Re:StackOverflow Update: 560M Pageviews a Month, 25 Servers by Nick-Craver on Tuesday July 22, 2014 @06:23AM (#8068186)
Worth noting: AWS also doesn't have Enterprise SQL available, so plan on buying that yourself. That's a not-so-insignificant piece of the overall cost to many people.
Re:StackOverflow Update: 560M Pageviews a Month, 25 Servers by superuser2 on Tuesday July 22, 2014 @02:05AM (#8067574)
How common is it to buy one extremely powerful database server? Wouldn't you need the high availability properties of something like a Galera cluster? Or are most people accepting the single point of failure at huge database server?

(I honestly don't know, it's just surprising.)

Re:StackOverflow Update: 560M Pageviews a Month, 25 Servers by emilv on Tuesday July 22, 2014 @03:00AM (#8067668)
While you do want HA you also want to keep things simple. You get great HA with just one backup machine. The times you lose two machines are so few that you might crash the site with software failures far more often anyway.

Databases have inherent locking problems that takes more resources to resolve with more machines, more so in some database systems than others. When scaling you often hit a point where you get down to macro logistics, so imagine a car highway: You can get more throughput by adding more lanes. But what that all cars are going to merge to one lane at some point in the middle of the trip (because of a tunnel, bridge or something: it's impossible to have more than one lane at that point)? Now you won't get any throughput benefits of the multiple lanes after all! Just more latency because you get queues up until the single lane and because of the queues all cars are going slow and need to accelerate on the single lane, so the average speed is low too. You are also having too much resources after the single lane because you can never fill all those lanes. It might be better to have one beefed-up single-lane road all the way that people can go fast on. Basically: Remove locks and you get better overall performance.

Yes, this is on the expense of HA. Yes, the costs of scaling up grows asymptotically faster than the costs of scaling out. So this is definitely a trade-off in some sense.

Re:StackOverflow Update: 560M Pageviews a Month, 25 Servers by jtseng on Friday August 01, 2014 @07:34PM (#8123402)
They do have a cluster: http://www.brentozar.com/archive/2012/09/microsoft-sql-serve...

They just run everything through one live database all the time under normal conditions. Personally, I think this is much simpler to manage than to be regularly spreading out queries over many queries.