Re:Applebot, the web crawler for Apple
by nemothekid on Wednesday May 06, 2015 @06:40AM
(#9497665)
Facebook blocks all bots except a select few to prevent site scraping - https://www.facebook.com/robots.txt
[ Parent
| on HN ]
There are no comments on this comment yet.