Re:Google Now Fills Out Forms & Crawls Results
by michaelneale on Friday April 11, 2008 @06:41PM
(#161444)
So its ok for them to do this? but if you do the same against google (scraping results) - presumably Bad Things happen?
[ Parent
| on HN ]
Re:Google Now Fills Out Forms & Crawls Results
by cstejerean on Friday April 11, 2008 @07:19PM
(#161476)
Is their search url listed in their robots.txt?
Re:Google Now Fills Out Forms & Crawls Results
by thorax on Friday April 11, 2008 @07:41PM
(#161491)
Seems strange that you wouldn't have just looked when asking that question (less characters to type at least). Lots of things disallowed.
Re:Google Now Fills Out Forms & Crawls Results
by Hexstream on Friday April 11, 2008 @11:07PM
(#161597)
I think it was a rhetorical question.
Re:Google Now Fills Out Forms & Crawls Results
by michaelneale on Sunday April 13, 2008 @12:45AM
(#162220)
Yes it was. But I should have been clearer for the type of literal argument starters that would tend to hang around here ;)
I wonder if google always honour robots.txt