News for hackers. Stuff that matters.
Re:The (humorous) dangers of Google's new deep search by neilc on Thursday April 17, 2008 @06:43PM (#166762)
It's not "their way of doing things", it is just the way the web works (per HTTP spec). Any crawler would have done the same thing in that situation -- that fact that it happened to be Google is merely coincidental. Given the scale at which they operate, you can't expect Google or any other web-scale crawler to be mind-readers.

There are no comments on this comment yet.