Re:Google Bot Attempts to Crawl Shortest URLs First
by bauchidgw on Saturday June 25, 2011 @11:16AM
(#2695809)
cant confirm this - sitemaps are used for discovery -> the urls listed in the sitemap get pushed into the 'discovered urls queue' then this queue is prioritized for crawling- and - if there are no other factors - the shorter urls get prioritized higher (as there is a bigger chance that a shorter url is a canonical version of a longer url - well, the chance is bigger then the other way round
[ Parent
| on HN ]
There are no comments on this comment yet.