Re:Avoiding bot detection: How to scrape the web without get...
by arp242 on Sunday October 31, 2021 @10:52PM
(#29062604)
By default Selenium exposes a few things in JS that are pretty trivial to detect, so you need to disable/hide that for starters. I don't know how easy or hard that is, but stock Selenium is a poor way to get around anti-bot stuff.
[ Parent
| on HN ]
Re:Avoiding bot detection: How to scrape the web without get...
by mhmmmmmm on Monday November 01, 2021 @04:16AM
(#29064164)
Its fairly trivial, some stuff needs a modified browser executable or some JavaScript magic but it wont take you longer than 2-5 Hours to bypass most of the heuristics, disabling the window.navigator.webdriver flag and getting a residential IP is on its own enough to get single click captchas every 1-2 tries for example. To be fair I haven't looked into it since 2019 but i doubt that its gotten much harder.