News for hackers. Stuff that matters.

A comment on What does a leaked Google memo reveal about the future of AI?. Parent comment

Re:What does a leaked Google memo reveal about the future of... by fomine3 on Monday May 15, 2023 @02:48AM (#35944582)
For websites that blocks all robots.txt-compliant bots except Google, Bing and some, Googlebots can crawl but CommonCrawl bots can't
[ Parent | on HN ]

There are no comments on this comment yet.