News for hackers. Stuff that matters.
Re:Common Crawl by visarga on Friday March 26, 2021 @11:35PM (#26599699)
The data is open they don't crawl inside password protected areas of the internet. Why shouldn't they be able to read data put into the open? If the authors didn't want their data openly accessible they should put it in a private place.
Re:Common Crawl by nodamage on Saturday March 27, 2021 @01:31AM (#26600165)
Except just because someone published something on the internet doesn't mean you have the legal right to reproduce and redistribute it yourself.
Re:Common Crawl by fjeifisjf on Saturday March 27, 2021 @08:47AM (#26601921)
That has nothing to do with reading data that the publisher does distribute.
Re:Common Crawl by nodamage on Sunday March 28, 2021 @06:53PM (#26615567)
They're not just reading data, they are redistributing it.