News for hackers. Stuff that matters.
Re:How to scrape anything on the web and not get caught by mmcwilliams on Sunday April 22, 2018 @08:09AM (#16895826)
I'm sort of on the other side of this problem, but I think an interesting counter-measure came up in a recent Show HN[0].

IP bans alone are probably not enough to stop a motivated scraper, or work longterm, so checking whether or not traffic is coming from a cloud provider and throwing up a captcha would be a significant hurdle, while allowing humans on cloud-hosted VPNs to pass through without much trouble.

[0] https://news.ycombinator.com/item?id=16868012

There are no comments on this comment yet.