News for hackers. Stuff that matters.
Re:Google Bot Attempts to Crawl Shortest URLs First by underdown on Saturday June 25, 2011 @02:56PM (#2696281)
"why would you crawl the site any other way?"

I could see crawling pages most likely to have changed first as those pages would most likely lead to fresh content.

There are no comments on this comment yet.