Re:Show HN: MrScraper – A visual web-scraping tool
by mellosouls on Friday February 10, 2023 @03:20PM
(#34745244)
That is a good reply generally, but this
Your assumption that blocking is somehow ethical by default is not unproblematic.
is itself an assumption.
The problem I'm concerned with is aggressive (either deliberately or ignorantly) crawling/scraping of non-commercial sites which often lack the financial resources to defend against activities enabled without apparent concern by tools like the site here.
If a site allows reasonable access in good faith, then subverting those limits and constraints for self-serving reasons is ethically dubious at best, and any service not addressing that while promising to enable that subversion should be questioned.
Your assumption that blocking is somehow ethical by default is not unproblematic.
is itself an assumption.
The problem I'm concerned with is aggressive (either deliberately or ignorantly) crawling/scraping of non-commercial sites which often lack the financial resources to defend against activities enabled without apparent concern by tools like the site here.
If a site allows reasonable access in good faith, then subverting those limits and constraints for self-serving reasons is ethically dubious at best, and any service not addressing that while promising to enable that subversion should be questioned.