News for hackers. Stuff that matters.
Re:Meta torrented & seeded 81.7 TB dataset containing copyri... by threeseed on Friday February 07, 2025 @03:55PM (#42977373)
From the docs:

“While Google won't crawl or index the content blocked by a robots.txt file”

They will show the URL if someone else has linked to it. But the content itself is not indexed.

There are no comments on this comment yet.