Re:Common Crawl
by visarga on Friday March 26, 2021 @11:35PM
(#26599699)
The data is open they don't crawl inside password protected areas of the internet. Why shouldn't they be able to read data put into the open? If the authors didn't want their data openly accessible they should put it in a private place.
[ Parent
| on HN ]
Except just because someone published something on the internet doesn't mean you have the legal right to reproduce and redistribute it yourself.
That has nothing to do with reading data that the publisher does distribute.