Re:Brave Search API now accepts agentic payments via the x40...
by cvadict on Friday November 13, 2026 @12:36PM
(#50486864)
> Presumably, if you pay and respect rate limits, you get to bypass the anti-scraping measures?
Far cheaper to just use your hyper-intelligence machines to bypass the anti-scraping measures than to pay everyone.
As of today you can only block the scattershot scrapers trying to slurp up everything at minimal effort/cost. Thankfully those make up the vast majority of traffic, so even dead-simple measures like Anubis block them. Whereas an advanced scraper is getting through any technical measures you can conceivably put in place (including the SOTA paid/cloud offerings). At the high end, the agents just drive regular browsers, mobile device, etc. running through a residential proxy / mobile CGNAT'd ip, solving captcha's in real-time, reasoning about the protection measures in place + adapting in real-time, etc. etc. They are virtually indistinguishable from humans. If you want humans to be able to see your content, the bots will too, as your average bot can now be far smarter + faster + adaptable than your average human.
Far cheaper to just use your hyper-intelligence machines to bypass the anti-scraping measures than to pay everyone.
As of today you can only block the scattershot scrapers trying to slurp up everything at minimal effort/cost. Thankfully those make up the vast majority of traffic, so even dead-simple measures like Anubis block them. Whereas an advanced scraper is getting through any technical measures you can conceivably put in place (including the SOTA paid/cloud offerings). At the high end, the agents just drive regular browsers, mobile device, etc. running through a residential proxy / mobile CGNAT'd ip, solving captcha's in real-time, reasoning about the protection measures in place + adapting in real-time, etc. etc. They are virtually indistinguishable from humans. If you want humans to be able to see your content, the bots will too, as your average bot can now be far smarter + faster + adaptable than your average human.