Show HN: PyDoll – Async Python scraping engine with native CAPTCHA bypass

Re:Show HN: PyDoll – Async Python scraping engine with nativ... by mrweasel on Tuesday June 10, 2025 @02:20PM (#44239712)

You know that the captcha is there to prevent you from doing e.g. automated data mining, depends on the site obviously. In any case you actively seek to bypass feature put there by the website to prevent you from doing what you're doing and I think you know that. Does that not give you any moral concerns?

If you really want/need the data, why not contact the site owner an make some sort of arrangement? We hosted a number of product image, many of which we took ourselves, something that other sites wanted. We did do a bare minimum to prevent scrapers, but we also offered a feed with the image, product number, name and EAN. We charged a small fee, but you then got either an XML feed or a CSV and you could just pick out the new additions and download those.

[ Parent | on HN ]

Re:Show HN: PyDoll – Async Python scraping engine with nativ... by lazyasciiart on Tuesday June 10, 2025 @04:01PM (#44240828)

Because Facebook isn’t open to making arrangements

[ Parent | on HN ]

Re:Show HN: PyDoll – Async Python scraping engine with nativ... by thalissonvs on Tuesday June 10, 2025 @05:11PM (#44241473)

I'm not actually bypassing the captcha with reverse engineering or anything like that, much less integrating with external services. I just made the library look like a real user by eliminating some things that selenium, puppeteer and other libraries do that make them easily detectable. You can still do different types of blocking, such as blocking based on IP address, rate limiting, or even using a captcha that requires a challenge, such as recaptchav2

[ Parent | on HN ]

Re:Show HN: PyDoll – Async Python scraping engine with nativ... by freehorse on Tuesday June 10, 2025 @07:37PM (#44242695)

> You can still do different types of blocking [...]

So, basically, make the internet hostile to everyone?

[ Parent | on HN ]