News for hackers. Stuff that matters.
Re:Minecraftwiki serving more traffic than Stackoverflow wit... by points on Friday October 15, 2010 @07:39PM (#1796805)
Is this the shape of things to come?

Comparisons of apples to oranges... It's not about traffic, it's about what sort of website you have, how dynamic/static it is, etc

A wiki likely has relatively few writes compared to reads, so caching should work very well.

That said, always nice to see people optimizing properly and using a sane number of servers.

Re:Minecraftwiki serving more traffic than Stackoverflow wit... by superjared on Friday October 15, 2010 @07:43PM (#1796819)
That's kind of the point. Yesterday Spolsky asked why Digg had 500 servers and less traffic than StackOverflow, yet StackOverflow had 5 servers.

False dichotomies huzzah!

Re:Minecraftwiki serving more traffic than Stackoverflow wit... by Keyframe on Friday October 15, 2010 @07:52PM (#1796831)
Yes, title is kind of a homage to that false dichotomy posted yesterday by Spolsky.
Re:Minecraftwiki serving more traffic than Stackoverflow wit... by bad_user on Saturday October 16, 2010 @10:48AM (#1798026)
Yes but StackOverflow is very similar to Digg:

you've got articles that have rating, you've comments to those articles that have rating, articles and comments are sorted according to rating, people earn karma from posting good comments / articles, everything has tags.

So personally I don't see a false dichotomy; even if Digg is more dynamic / complex ... WTF are they doing with those 500 servers?

Re:Minecraftwiki serving more traffic than Stackoverflow wit... by qeorge on Saturday October 16, 2010 @06:31PM (#1799037)
Digg is more like Facebook and Twitter than StackOverflow. Each of Digg's logged in users gets their own "News Feed" based on the users they follow/friended/are most similar to.

StackOverflow on the other hand is much simpler - questions and responses, plus users and voting. AFAIK there's no collaborative filtering going on at SO, be it user or item based.

I don't know why Digg needs so many boxen, but I did find Spolsky's comparison disingenuous.

Re:Minecraftwiki serving more traffic than Stackoverflow wit... by points on Friday October 15, 2010 @07:44PM (#1796821)
Well TBH 500 is ridiculous overkill. Even 100 would be pushing it.
Re:Minecraftwiki serving more traffic than Stackoverflow wit... by Goladus on Saturday October 16, 2010 @07:30AM (#1797760)
Whether it's overkill or not depends entirely on what the servers are being used for. Without actually working there it's hard for one to say for sure.
Re:Minecraftwiki serving more traffic than Stackoverflow wit... by points on Saturday October 16, 2010 @12:43PM (#1798257)
We know what the service does. We have that much data - we can see what users are allowed to do on the website, and what they cannot do.

And for the level of content, interactivity, media, etc etc, I can say that 500 is silly.

Re:Minecraftwiki serving more traffic than Stackoverflow wit... by Goladus on Monday October 18, 2010 @06:47AM (#1802913)
You don't know what each server is doing. They might have 10 servers being used by an internal marketing analytics team, with 40 support servers for development, QA, testing, and disaster recovery of those specific services.

What sort of computing resources do you think it took for Google to develop the autopilot car? Would you have been able to determine that by looking at their homepage and the services they provide? No. That's the point. I'm not suggesting that Digg is doing anything so interesting, but most of those 500 servers are almost certainly NOT being used to support their website directly, they're being used by the business for other things.

Re:Minecraftwiki serving more traffic than Stackoverflow wit... by citricsquid on Friday October 15, 2010 @08:19PM (#1796882)
The title was supposed to be a serious representation of our traffic numbers (our amount of servers wasn't supposed to be part of the comparison, the poster here is a slippery snake) but we can pretend it was if you like, nobody likes joel so it fits in well.
Re:Minecraftwiki serving more traffic than Stackoverflow wit... by ericflo on Friday October 15, 2010 @08:11PM (#1796866)
I liked this submission simply because it demonstrates the ridiculousness of such claims.
Re:Minecraftwiki serving more traffic than Stackoverflow wit... by fauigerzigerk on Saturday October 16, 2010 @04:06AM (#1797586)
Spolsky compared two rather more similar sites though. This one is a wiki where most of the content is static, served out of cache. Still, I agree that such comparisons are on a very weak footing. A single feature could easily kill any valid comparison.
Re:Minecraftwiki serving more traffic than Stackoverflow wit... by ojbyrne on Friday October 15, 2010 @08:47PM (#1796940)
Thankfully it wasn't actually less traffic, it was only 4x (maybe 5x) the traffic, with 10x the servers.
Re:Minecraftwiki serving more traffic than Stackoverflow wit... by rythie on Friday October 15, 2010 @08:22PM (#1796889)
Also, it's MediaWiki so all the optimisation work was done by someone other people, to make Wikipedia work.
Re:Minecraftwiki serving more traffic than Stackoverflow wit... by citricsquid on Friday October 15, 2010 @08:23PM (#1796892)
Mediawiki is fantastic software, we'd be able to operate on a lot less hardware if it was just mediawiki, but because of the forum we had to boost everything up. For anyone who ever considers using phpbb for anything serious: please don't.
Re:Minecraftwiki serving more traffic than Stackoverflow wit... by hartror on Friday October 15, 2010 @09:25PM (#1797024)
Suggestions for good alternatives with good scaling?
Re:Minecraftwiki serving more traffic than Stackoverflow wit... by citricsquid on Friday October 15, 2010 @09:57PM (#1797092)
We're looking at a couple of alternatives and I can report back with our findings when we know enough, right now we're looking at vbulletin which powers some of the larger forums, it is apparently very good if you're willing to strip out the poor parts (apparently search is terrible).

It looks as if the best approach is to roll your own, phpbb seems to be designed with the smaller user in mind, so while routing every single file through file.php for easy processing might work well for Johnny and his friends forum, once you hit a large scale it becomes rather a pain.

So yeah, no idea, we'll find out soon though, I'll report back when I know :-)

Re:Minecraftwiki serving more traffic than Stackoverflow wit... by sandGorgon on Saturday October 16, 2010 @12:30AM (#1797358)
Any chance you could post your configuration files somewhere ? (you know PHP, varnish, etc.) It'd be great to grok those.
Re:Minecraftwiki serving more traffic than Stackoverflow wit... by dmak on Saturday October 16, 2010 @12:09AM (#1797336)
Where can I stay posted with your findings?
Re:Minecraftwiki serving more traffic than Stackoverflow wit... by jshen on Friday October 15, 2010 @08:28PM (#1796902)
I think there is an important lesson in this even if it's apples to oranges. Often in my career I've been in a debate with a non engineer (product person, ceo, etc) about why a certain features sounds good to them but a variation of it which provides most of what they want is so much better because I can keep the page mostly static. Sometimes I've won that debate and sometimes I lost. Being able to hold up examples, see such and such wiki serves x million pages with 3 servers because it's mostly static vs ...