News for hackers. Stuff that matters.
Re:Perplexity is using stealth, undeclared crawlers to evade... by Den_VR on Monday August 04, 2025 @10:26AM (#44786281)
You can limit access, sure: with ACLs, putting content behind login, certificate based mechanisms, and at the end of the day -a power cord-.

But really, controlling which automated retrieval tools are allowed has always been more of a code of honor than a technical control. And that trust you mention has always been broken. For as long as I can remember anyway. Remember LexiBot and AltaVista?

There are no comments on this comment yet.