News for hackers. Stuff that matters.

A comment on Perplexity is using stealth, undeclared crawlers to evade no-crawl directives. Parent comment

Re:Perplexity is using stealth, undeclared crawlers to evade... by TZubiri on Monday August 04, 2025 @12:39PM (#44788224)
In that case the llm would be a user-agent, quite distinct from scraping without a specific user request.

This is well defined in specs and ToS, not quite a gray area

[ Parent | on HN ]

There are no comments on this comment yet.