Re:Who exactly is crawling my site?
by droidlabour on Friday October 18, 2013 @05:11PM
(#6574041)
What if a bot uses user-agent to look like it came from an end-user browser?
[ Parent
| on HN ]
Re:Who exactly is crawling my site?
by jessaustin on Friday October 18, 2013 @10:48PM
(#6575134)
It probably wouldn't act like a human-operated browser, even if it said it was.