Avoiding Webscraping Throttling Using Python and Tor as a Proxy

Re:Avoiding Webscraping Throttling Using Python and Tor as a... by vbezhenar on Saturday July 13, 2019 @05:03PM (#20430150)

Another problem is that blocking tor nodes is very easy. And with abusing tor nodes as free proxies, it makes a huge disservice to the Tor network, because more websites will ban it and ordinary users won't be able to use it.

[ Parent | on HN ]

Re:Avoiding Webscraping Throttling Using Python and Tor as a... by beardog on Saturday July 13, 2019 @06:03PM (#20430448)

This problem is being partly addressed with Privacy Pass. The addon lets you bypass recaptcha including cloudflare recaptcha minus one initial solve per session. Of course this doesn't account for all blocks and of course some malicious bots bypass recaptcha anyway using AI/speech recognition or human labor farms, but it will make the ecosystem better for everyone.

(note: it is not a captcha solving service, its a collaborative project by cloudflare, google and others)

https://privacypass.github.io/

[ Parent | on HN ]