Re:Google Bot Attempts to Crawl Shortest URLs First
by abrudtkuhl on Saturday June 25, 2011 @10:59AM
(#2695770)
It starts at the top of your sitemap - which are likely shorter URLs
[ Parent
| on HN ]
Re:Google Bot Attempts to Crawl Shortest URLs First
by bauchidgw on Saturday June 25, 2011 @11:16AM
(#2695809)
cant confirm this - sitemaps are used for discovery -> the urls listed in the sitemap get pushed into the 'discovered urls queue' then this queue is prioritized for crawling- and - if there are no other factors - the shorter urls get prioritized higher (as there is a bigger chance that a shorter url is a canonical version of a longer url - well, the chance is bigger then the other way round