News for hackers. Stuff that matters.
Re:Meta torrented & seeded 81.7 TB dataset containing copyri... by tobyhinloopen on Friday February 07, 2025 @07:52AM (#42971998)
a) If you don't have a robots.txt, you're indexed by default. It's opt-out, not opt-in. If you do nothing, you're being indexed.
Re:Meta torrented & seeded 81.7 TB dataset containing copyri... by antiframe on Friday February 07, 2025 @12:50PM (#42975418)
It's an opt-out of an opt-in. If you run a webserver hosting your files, you already opted-in to people accessing that data. If you then don't go ahead an configure it properly, that's not exactly "opt-out" anymore. By default your files are not accessible to the network, you have to first opt-in to serving them.
Re:Meta torrented & seeded 81.7 TB dataset containing copyri... by tobyhinloopen on Saturday February 08, 2025 @03:44AM (#42981510)
Google makes a copy of your data and serves that data to users before they visit your site.

Also google cache allows users to get a copy of your site without visiting your site.

Why can they republish your data while we cannot? Why do we have to opt-out?