News for hackers. Stuff that matters.

A comment on Perplexity is using stealth, undeclared crawlers to evade no-crawl directives. Parent comment

Re:Perplexity is using stealth, undeclared crawlers to evade... by jychang on Monday August 04, 2025 @08:35PM (#44793026)
Who cares what Hacker News wants? You’re not obliged to participate in discussion.

Am I supposed to spend money on Amazon.com when I visit the website just because Amazon wants me to?

[ Parent | on HN ]
Re:Perplexity is using stealth, undeclared crawlers to evade... by egypturnash on Monday August 04, 2025 @11:29PM (#44794036)
If most people quit spending money on Amazon then Amazon stops being worth running.

If most people stop discussing things on HN, and the discussion is indeed one of the major reasons it’s kept running, then HN stops being worth running.

[ Parent | on HN ]
Re:Perplexity is using stealth, undeclared crawlers to evade... by AuthAuth on Tuesday August 05, 2025 @12:34AM (#44794380)
Whats the point of a human coming to a site if all the threads and empty and its front page is a glorified RSS feed for lazy peoples AI agents?
[ Parent | on HN ]
Re:Perplexity is using stealth, undeclared crawlers to evade... by AnthonBerg on Tuesday August 05, 2025 @01:12AM (#44794530)
Who cares what you want?
[ Parent | on HN ]
Re:Perplexity is using stealth, undeclared crawlers to evade... by danlitt on Tuesday August 05, 2025 @03:15AM (#44795161)
Most humans place the desires of human beings over the desires of companies.
[ Parent | on HN ]
Re:Perplexity is using stealth, undeclared crawlers to evade... by lenkite on Saturday August 09, 2025 @10:59PM (#44852452)
Indeed. But that is a false equivalence - this is conflict of desires between small companies and creators and an AI-corp where the AI-corp wants to steal their content and give it to users with their shop branding.
[ Parent | on HN ]
Re:Perplexity is using stealth, undeclared crawlers to evade... by butlike on Wednesday August 06, 2025 @09:05AM (#44811432)
It was a corollary example
[ Parent | on HN ]
Re:Perplexity is using stealth, undeclared crawlers to evade... by noboostforyou on Wednesday August 06, 2025 @03:59PM (#44816949)
> You’re not obliged to participate in discussion.

Are website owners obligated to serve content to AI agents and/or LLM scrapers?

[ Parent | on HN ]