News for hackers. Stuff that matters.
Re:Stack Overflow Architecture Update - Now At 95 Million Pa... by moe on Thursday March 03, 2011 @02:53PM (#2285491)
Hm. I love Stack Overflow but architecture only starts to become interesting at 4-5 times their traffic.

95M/mo translates to a mere 36/sec avg. A single web- and db-server will handle that without breaking a sweat, although you of course want some more machines for peaks, redundancy and comfort.

Re:Stack Overflow Architecture Update - Now At 95 Million Pa... by citricsquid on Thursday March 03, 2011 @03:01PM (#2285521)
36/sec but then they're doing 800 http requests per second, not sure what could cause such a big difference (lots of ajax stuff?) but I think 800/sec is an interesting thing to see them deal with, that's something in the region of 1bn a month, which fits your requirement for interesting-ness :p
Re:Stack Overflow Architecture Update - Now At 95 Million Pa... by icey on Thursday March 03, 2011 @03:28PM (#2285618)
Perhaps the 800 req/second number is a peak figure? 36 requests a second average isn't meaningful when your traffic is spiky.
Re:Stack Overflow Architecture Update - Now At 95 Million Pa... by moe on Thursday March 03, 2011 @06:57PM (#2286294)
which fits your requirement for interesting-ness

Sorry, the threshold of interestingness for serving static assets is even higher. ;)

A single nginx on a moderate host will barely warm up before ~10k reqs/sec. The network tends to be the bottleneck here. Anyways, ~800/s should be doable from a $10 VPS.

Re:Stack Overflow Architecture Update - Now At 95 Million Pa... by dillona on Thursday March 03, 2011 @07:35PM (#2286402)
I don't think they're paying all this money for the fun of it
Re:Stack Overflow Architecture Update - Now At 95 Million Pa... by moe on Thursday March 03, 2011 @08:09PM (#2286499)
Hm, I'm not sure I follow. What money do you mean?
Re:Stack Overflow Architecture Update - Now At 95 Million Pa... by alextgordon on Thursday March 03, 2011 @08:24PM (#2286539)
They have 10 web servers. If they could replace all that with a VPS or two, they would.
Re:Stack Overflow Architecture Update - Now At 95 Million Pa... by moe on Thursday March 03, 2011 @10:34PM (#2286893)
According to the article only 3 are dedicated to StackOverflow which sounds about right to me.
Re:Stack Overflow Architecture Update - Now At 95 Million Pa... by IDisposableHero on Friday March 04, 2011 @09:57AM (#2288191)
The VPS figure was for serving static files. So, a lot of the stackoverflow content isn't static.
Re:Stack Overflow Architecture Update - Now At 95 Million Pa... by jonknee on Thursday March 03, 2011 @06:23PM (#2286200)
Probably a combination of other page resources (images, JS, stylesheets, etc), bots and peak access times.
Re:Stack Overflow Architecture Update - Now At 95 Million Pa... by siavash on Thursday March 03, 2011 @07:29PM (#2286379)
We (blocket.se) are hitting almost 1b/mo and I can assure you that it's irrelevant wether the average p/v per second can be handled by some basic hardware or not. During peak hours it's an order of magnitude difference. Worth mentioning though that we are tightly focused in only one country, and probably have larger variations during peak hours than stack overflow does. I'm not sure what our traffic looks like combined with our other local sites.
Re:Stack Overflow Architecture Update - Now At 95 Million Pa... by moe on Thursday March 03, 2011 @10:57PM (#2286939)
If your peak hours exceed average by an order of magnitude then you have oddly shaped traffic. Most sites follow a bell curve where regular peak hours range around factor 2-3 above average.

Either way I didn't mean to discount their efforts. Was just trying to point out that their architecture is not very interesting from a performance point of view, yet.

Re:Stack Overflow Architecture Update - Now At 95 Million Pa... by PostOnce on Thursday March 03, 2011 @06:32PM (#2286223)
I wish I had to figure out how to deal with a "mere 36/sec avg" number of visitors. I'd sell my company and retire.
Re:Stack Overflow Architecture Update - Now At 95 Million Pa... by IDisposableHero on Friday March 04, 2011 @04:56AM (#2287583)
architecture only starts to become interesting at 4-5 times their traffic.

Which suggests that even sites with a medium-to-large internet presence, which dominatre thier niche, can do so with "uninteresting" architecture.

Re:Stack Overflow Architecture Update - Now At 95 Million Pa... by moe on Monday March 07, 2011 @07:22AM (#2296890)
Which suggests that even sites with a medium-to-large internet presence [...] can do so with "uninteresting" architecture

Absolutely. Hardware is evolving so fast (Moore's Law & friends) that we humans have a hard time to keep up mentally.

Go back six years in time and the traffic they're dealing would have required roughly 24 servers, instead of the 3 that they have today.

This is of course a rough extrapolation and six years seems like an eternity on the internet-calendar.

However we're quickly approaching a point where there's only two scales left to worry about: "normal" and "web-scale", with only a few hundred sites falling into the latter category.