News for hackers. Stuff that matters.

A comment on Perplexity is using stealth, undeclared crawlers to evade no-crawl directives. Parent comment

Re:Perplexity is using stealth, undeclared crawlers to evade... by autoexec on Tuesday August 05, 2025 @12:20PM (#44800067)
If people were forced to pay for websites by the http request people would demand that websites stop loading a ton of externally hosted JS, stop filling sites with ads, and would demand that websites actually have content worth the price.

There are so many links I click on these days that are such trash I'd be demanding refunds constantly.

[ Parent | on HN ]
Re:Perplexity is using stealth, undeclared crawlers to evade... by trhway on Tuesday August 05, 2025 @02:20PM (#44802070)
>There are so many links I click on these days that are such trash

That is why AI "summarization" becomes a necessary intermediate layer. You'd not see nor trash nor ads, and thus the payment instead of being exposed to the ads. AI saves the Internet :)

[ Parent | on HN ]