News for hackers. Stuff that matters.
Re:Detecting Chrome headless by always_good on Saturday August 05, 2017 @02:17PM (#14937063)
Also, what I've learned is how little regard for your site your scrapers often have, scraping as aggressively as possible.

You're just not always in a place to scale to the abuse or build something more complex than some simple heuristic filters.

Re:Detecting Chrome headless by josteink on Sunday August 06, 2017 @09:49AM (#14941298)
> what I've learned is how little regard for your site your scrapers often have, scraping as aggressively as possible.

Often? Based on what data?

I find it much more likely you only often notice aggressive scrapers. That however tells you nothing about the behavior of the average web scraper or web scrapers in general.

Re:Detecting Chrome headless by tyingq on Sunday August 06, 2017 @12:04PM (#14941824)
The system encourages it. Ingress data is cheap, and so many scrapers just default to high frequency.