News for hackers. Stuff that matters.
Re:Google's robots.txt by kercker on Sunday February 23, 2014 @01:09AM (#7285108)
I'm sure that Baidu respects robots.txt.

This site coolshell.cn is boycotting baidu, so it tells Baiduspider that it doesn't want to be indexed by baidu.

Baidu respects this and doesn't index anything from this website.

There are no comments on this comment yet.