Re:Googlebot snarfing URLs from the ether?
by stray on Sunday June 02, 2013 @10:59AM
(#5808728)
Did you access that URL with Google Chrome mere seconds before googlebot miraculously found it?
[ Parent
| on HN ]
Re:Googlebot snarfing URLs from the ether?
by galacticvoid on Sunday June 02, 2013 @11:05AM
(#5808754)
Or does the browser your using have instance results from the address bar and the search engine is Google?
Re:Googlebot snarfing URLs from the ether?
by colanderman on Sunday June 02, 2013 @12:58PM
(#5809201)
Yes, though I've tried testing for this, but I only see requests from my browser (Opera) for incomplete URLs.
Re:Googlebot snarfing URLs from the ether?
by colanderman on Sunday June 02, 2013 @12:57PM
(#5809192)
No, I'm using Opera (see the log). It's possible Opera is sending such info as well (it uses Google for autocompletion), but I've tried reproducing this and I don't see evidence of it. (I do see autocompletion requests, but never for complete URLs.)
That is surprising and somewhat troubling.
Wait, are you saying the Google Chrome will send URLs back to Google for indexing? I did not know this
I'm not saying it as a statement of fact. But rather as a plausible explanation for what OP experienced.
But if I were google I would definitely give people free tools with which they could perform, with actual human intelligence, tasks I had found difficult.