Re:AI bots are now outperforming humans in solving CAPTCHAs
by rezonant on Sunday December 17, 2023 @05:14AM
(#38671798)
I guess it depends on how you're scraping. For general web crawling, simply implementing a response time based crawl back off per origin and identifying yourself appropriately in User Agent goes a long way. If you are instead automating Facebook's SPA to pull comments for analysis, then yeah you need to emulate a human, because that's not how they intend you to do it.
With coffee N(1,5 minutes, 20 seconds), toilet N(4 minutes, 30 seconds), ...