News for hackers. Stuff that matters.

A comment on Quora Blocks Startup Search Engines. Parent comment

Re:Quora Blocks Startup Search Engines by palish on Thursday January 27, 2011 @10:19PM (#2150648)
"You can't change the crawl rate for sites that are not at the root level—for example, www.example.com/folder."

What if Googlebot starts at a non-root location (via a link from another site)?

Will it always start at root?

[ Parent | on HN ]
Re:Quora Blocks Startup Search Engines by terryjsmith on Thursday January 27, 2011 @10:56PM (#2150734)
I think they just mean that it's a whole site/domain action; that you can't change crawl rate for a sub-directory based site ala Wordpress MU/Multisite with directories.
[ Parent | on HN ]