News for hackers. Stuff that matters.

A comment on Ask HN: What are best tools for web scraping?.

Re:Ask HN: What are best tools for web scraping? by frausto on Tuesday November 14, 2017 @12:00PM (#15696826)
Been getting blocked by recaptcha more and more, do any of these tools handle dealing with that or workarounds by default? Tried routing through proxies and swapping IP addresses, slowing down, etc... Any specific ways people get around that?
[ Parent | on HN ]
Re:Ask HN: What are best tools for web scraping? by jakubbalada on Tuesday November 14, 2017 @12:05PM (#15696886)
You can use services like Anti-captcha [1]

We have a public API on Apify for that [2]

[1] https://anti-captcha.com/mainpage

[2] https://www.apify.com/petr_cermak/anti-captcha-recaptcha

[ Parent | on HN ]
Re:Ask HN: What are best tools for web scraping? by levi_n on Tuesday November 14, 2017 @12:19PM (#15697044)
The excepted answer on this stack overflow question[1] might help. tl;dr is to build your own chromedriver, but with renamed variables.

[1] https://stackoverflow.com/a/41220267/4079962

[ Parent | on HN ]