News for hackers. Stuff that matters.

A comment on Perplexity is using stealth, undeclared crawlers to evade no-crawl directives. Parent comment

Re:Perplexity is using stealth, undeclared crawlers to evade... by Chris2048 on Monday August 04, 2025 @10:34AM (#44786378)
If I put time and effort into a food recipe should I (get) compensation?

the answer is apparently "no", and I don't really how recipe books have suffered as a result of less gatekeeping.

"How will the internet work"? Probably better in some ways. There is plenty of valuable content on the internet given for free, it's being buried in low-value AI slop.

[ Parent | on HN ]
Re:Perplexity is using stealth, undeclared crawlers to evade... by Workaccount2 on Monday August 04, 2025 @11:17AM (#44787035)
You understand that HN is ad supported too, right?
[ Parent | on HN ]
Re:Perplexity is using stealth, undeclared crawlers to evade... by Chris2048 on Monday August 04, 2025 @11:32AM (#44787249)
No, I don't.

But what is your point? Is the value in HN primarily in its hosting, or the non-ad-supported community?

[ Parent | on HN ]
Re:Perplexity is using stealth, undeclared crawlers to evade... by Workaccount2 on Monday August 04, 2025 @01:19PM (#44788801)
Outside of Wikipedia, I'm not sure what content you are thinking of.

Taking HN as a potential one of these places, it doesn't even qualify. HN is funded entirely to be a place for advertising ycombinator companies to a large crowd of developers. HN is literally a developer honey pot that they get exclusive ad rights to.

[ Parent | on HN ]