News for hackers. Stuff that matters.

A comment on Google Says It'll Scrape Everything You Post Online for AI. Parent comment

Re:Google Says It'll Scrape Everything You Post Online for AI by palata on Monday July 03, 2023 @07:03PM (#36579944)
Does anyone respect robots.txt files?
[ Parent | on HN ]
Re:Google Says It'll Scrape Everything You Post Online for AI by scarface_74 on Monday July 03, 2023 @07:06PM (#36579967)
Any history of Google not respecting it?
[ Parent | on HN ]
Re:Google Says It'll Scrape Everything You Post Online for AI by tteggel on Tuesday July 04, 2023 @12:07PM (#36588523)
Google does not respect robots.txt if a page is linked to from other sites

https://developers.google.com/search/docs/crawling-indexing/...

[ Parent | on HN ]
Re:Google Says It'll Scrape Everything You Post Online for AI by djmips on Wednesday July 05, 2023 @06:34AM (#36598377)
That doesn't say the content but only that it is indexed.
[ Parent | on HN ]