News for hackers. Stuff that matters.

A comment on Google's robots.txt. Parent comment

Re:Google's robots.txt by yeukhon on Sunday February 23, 2014 @12:01AM (#7284999)
> Notice: Crawling Facebook is prohibited unless you have express written

Wow, really? Who put up this sign?

[ Parent | on HN ]
Re:Google's robots.txt by phinnaeus on Sunday February 23, 2014 @01:12AM (#7285111)
Someone with 80 character lines enforced in their editor? Here's the second line...

> permission. See: http://www.facebook.com/apps/site_scraping_tos_terms.php

[ Parent | on HN ]