Re:Meta's Muse is fantastic for web scraping
by pcthrowaway on Friday October 02, 2026 @01:50AM
(#49930196)
There is no detection method that will prevent AI from accessing systems without also blocking humans. The only thing we can do at this point is throttling.
[ Parent
| on HN ]
Re:Meta's Muse is fantastic for web scraping
by kees99 on Friday October 02, 2026 @02:06AM
(#49930295)
Agreed on inevitable collateral blockage of real people using real "headed" browser. I'm getting a ton of that already, personally.
Throttling is poor help though. Mass scrapers are using "residential proxy" loophole + rotating UA and other attributes. You can't throttle somebody without identifying them. Unless you're talking about a global rate-limit.
Re:Meta's Muse is fantastic for web scraping
by pcthrowaway on Friday October 02, 2026 @02:13AM
(#49930334)
throttling based on sessions kind of works; we're headed in the direction that sites like Reddit will probably prevent logged-out users from viewing threads (as they already do with mobile devices)
Once the LLMs create sockpuppets to get around that, the web services will need to resort to profiling users more aggressively so that they know which actual human an account corresponds to.
If someone has a malicious browser extension that uses their session to scrape Reddit then, they're probably going to see significant usage obstacles.
We are headed to a very user-hostile place.