Re:Perplexity is using stealth, undeclared crawlers to evade...
by mrinterweb on Monday August 04, 2025 @07:09PM
(#44792385)
Is a perplexity visit not cached and shared between users who perform a similar search? I don't know much about perplexity, but I'd be surprised if scraped results weren't used to serve multiple searches and users. By bypassing the no-crawl directive, that is a violation of the website's expressed request. I think it is different if individual users chooses to bypass certain things on a website, but for a company to choose to do it is another story.
[ Parent
| on HN ]
There are no comments on this comment yet.