News for hackers. Stuff that matters.
Re:Bing search results showing up in Google by benologist on Saturday February 05, 2011 @01:39PM (#2183644)
They proved that Microsoft uses clickstream data to rank websites, and in 7% of manufactured cases that's all the data they have?

I think this is exactly as petty and silly as last week's news, and now they get to spend a week explaining how this occurred and that they do obey robots.txt.

Re:Bing search results showing up in Google by moultano on Saturday February 05, 2011 @02:48PM (#2183861)
Everyone else in this thread has already explained that microsoft is not properly using robots.txt by having case insensitive urls, hence why these urls were indexed.
Re:Bing search results showing up in Google by jeroen on Saturday February 05, 2011 @03:24PM (#2183953)
"Be strict in what you emit, liberal in what you accept."

Bing failed the first part, Google the second. Both should fix that.

Re:Bing search results showing up in Google by benologist on Saturday February 05, 2011 @03:10PM (#2183930)
The Bing context might suck for you guys, but this is your problem - case insensitivity is every Windows server, not just the Bing website.

Why would anyone hosted on Windows have to specify every possible spelling variation to keep search engines out of a folder or file?

Here's another example:

http://www.ifma.org/robots.txt

These guys are disallowing /pv/

http://www.google.com/search?sourceid=chrome&ie=UTF-8...

You guys are indexing /PV/

Re:Bing search results showing up in Google by andrewcooke on Saturday February 05, 2011 @03:19PM (#2183941)
Maybe I'm misunderstanding your argument, but I think you're confusing the Windows file system with the URLs that a web service provides.
Re:Bing search results showing up in Google by benologist on Saturday February 05, 2011 @03:39PM (#2183993)
The problem exists when that web service is on Windows - ASP.NET, Cold Fusion, static html sites, probably a negligible percent of PHP sites etc -

/pv is /PV is /pV is /Pv

Re:Bing search results showing up in Google by mattmanser on Saturday February 05, 2011 @02:38PM (#2183829)
Except that this time it isn't Microsoft themselves crying foul.