News for hackers. Stuff that matters.
Re:Stack Overflow hit 1,000,000 questions last night by robgough on Wednesday October 13, 2010 @12:13PM (#1787632)
This is great news, if for no other reason that prowling google for answers is 100%[1] more effective than it used to be, as most searches lead me straight to stackoverflow.

[1] well, 100%-ish. I've not measured this. Nor am I going to.

Re:Stack Overflow hit 1,000,000 questions last night by markkanof on Wednesday October 13, 2010 @02:47PM (#1788330)
There seem to be a number of sites that just poorly mirror content from StackOverflow. They usually have the question and then maybe some answers, but don't usually include comments to answers or even which answer was marked as correct.

It's very frustrating when they come up above StackOverflow in Google searches. I end up slowing down a lot to make sure I don't click on those results. Sure I could just go straight to StackOverflow which I often do, but there are other good sources of programming knowledge on the internet that I want to be able to find too.

These mirror sites are also frustrating because they are blatantly leaching off of the work of both the StackOverflow developers and the StackOverflow community. Kind of makes me wish Google did have a blacklist for these cases.

Re:Stack Overflow hit 1,000,000 questions last night by spolsky on Wednesday October 13, 2010 @03:32PM (#1788525)
When we find out about these cases, we ask them to conform to our creative commons license agreement by linking back to stackoverflow.com, in hopes that ultimately this will teach Google that they are non-canonical and rank them lower than us.
Re:Stack Overflow hit 1,000,000 questions last night by brc on Thursday October 14, 2010 @01:35AM (#1789926)
That might be so, but right now there's about three of them and they clog up the search results with essentially the same answer but lower quality for the reasons mentioned above. So far I've not seen any that outrank stackoverflow, but it's only a matter of time before one does some blackhat and jumps for certain questions.

You say that they link back, but in the cases I've seen, they link back with a slightly different Url form to the one you use.

For example : http://stackoverflow.com/questions/1037925/recreate-stack-tr... this question appears on scraping sites (not going to name them for fear of feeding them traffic), but they link back with this Url: http://stackoverflow.com/questions/1037925 - which doesn't resolve back to the longer version. I know there's a canonical link on it (for the longer url) but I'm wondering if this strategy is working or not.

Maybe it's time to start rejecting some bots crawling the site, or at least to slow them right down.

It's a problem for Google and for SO, so someone from both places should probably be talking about it. Because in the end, SO starts looking spammy because the others are spammy and scrape the content, which is exactly the opposite case of how things should be. The spam sites are just cloning SO and shoving crappy ads everywhere, it's not high quality content. When you ask a question, you want 5 different answers from 5 different site, not the same answer spammed 5 times back at you.

Re:Stack Overflow hit 1,000,000 questions last night by cdr on Wednesday October 13, 2010 @01:19PM (#1787963)
I haven't noticed too much change in Google; I generally just go straight to Stackoverflow first.