News for hackers. Stuff that matters.

A comment on The State of Web Scraping 2022. Parent comment

Re:The State of Web Scraping 2022 by RobSm on Wednesday January 12, 2022 @02:18PM (#29911545)
No. robots.txt is not something that is defined and enforced by the law. Just because someone came up with some 'recommendation' like robots.txt does not mean this is the law
[ Parent | on HN ]
Re:The State of Web Scraping 2022 by shukantpal on Wednesday January 12, 2022 @03:30PM (#29912300)
> Legal? Yes. Parasitic behavior? Also yes.

You failed to make a meaningful counterpoint; the legal/ethical distinction was made clear in the parent post.

[ Parent | on HN ]
Re:The State of Web Scraping 2022 by LamaOfRuin on Wednesday January 12, 2022 @04:11PM (#29912721)
As a matter of fact, robots.txt is a well understood expression of intent which is legally meaningful in a lot of contexts.
[ Parent | on HN ]