News for hackers. Stuff that matters.

A comment on Web Scraping Is Vital to Democracy.

Re:Web Scraping Is Vital to Democracy by notahaterbut on Friday December 04, 2020 @10:24AM (#25303242)
Web-scraping can do harm to a business if the scraper does not implement things like timeouts or sleeps between processes.

Websites and businesses do not factor bandwidth for the rogue scraper.

[ Parent | on HN ]
Re:Web Scraping Is Vital to Democracy by PestoDiRucola on Friday December 04, 2020 @10:34AM (#25303378)
You can easily block a harmful scraper like that from the website owner's side.
[ Parent | on HN ]
Re:Web Scraping Is Vital to Democracy by kingdomcome50 on Friday December 04, 2020 @03:47PM (#25307692)
You make the target. I’ll make the bot. Let’s see who wins.

In all seriousness, your suggestion is woefully naive in the face of any serious entity.

[ Parent | on HN ]
Re:Web Scraping Is Vital to Democracy by notahaterbut on Friday December 04, 2020 @10:44AM (#25303530)
what if the scraper uses ip rotation or serverless workers ?
[ Parent | on HN ]
Re:Web Scraping Is Vital to Democracy by kordlessagain on Friday December 04, 2020 @10:42AM (#25303510)
This only applies to crawlers who are set to recurse into URLs.
[ Parent | on HN ]