News for hackers. Stuff that matters.

A comment on Perplexity is using stealth, undeclared crawlers to evade no-crawl directives. Parent comment

Re:Perplexity is using stealth, undeclared crawlers to evade... by rustc on Monday August 04, 2025 @12:00PM (#44787698)
> Should curl be considered a bot too? What's the difference?

Perplexity definitely does:

    $ curl -sI https://www.perplexity.ai | head -1
    HTTP/2 403
[ Parent | on HN ]

There are no comments on this comment yet.