News for hackers. Stuff that matters.
Re:Yandex can now search for Facebook posts by mapleoin on Tuesday January 14, 2014 @11:01AM (#7057691)
They give the Internet Archive the most access, but oddly go out of their way to block their TOS and privacy policy from them and not anyone else. Sneaky.

It's actually the opposite. They allow IA access to their terms and policy, but block everything else:

User-agent: ia_archiver

Allow: /about/privacy

Allow: /full_data_use_policy

Allow: /legal/terms

Allow: /policy.php

Disallow: /

Re:Yandex can now search for Facebook posts by jonknee on Tuesday January 14, 2014 @11:03AM (#7057706)
Duh, all the others were Disallow statements and I just missed that the IA ones were Allow. Thanks for the tip.