News for hackers. Stuff that matters.

A comment on Perplexity is using stealth, undeclared crawlers to evade no-crawl directives. Parent comment

Re:Perplexity is using stealth, undeclared crawlers to evade... by ivape on Monday August 04, 2025 @11:14AM (#44786993)
Or websites can monetize their data via paid apis and downloadable archives. That's what makes Reddit the most valuable data trove for regular users.
[ Parent | on HN ]
Re:Perplexity is using stealth, undeclared crawlers to evade... by ccgreg on Monday August 04, 2025 @03:29PM (#44790383)
I don't think Reddit pays the people who voluntarily write Reddit content. Valuable to Reddit, I guess.
[ Parent | on HN ]
Re:Perplexity is using stealth, undeclared crawlers to evade... by p3rls on Monday August 04, 2025 @10:20PM (#44793667)
so many braindead takes in this thread but expecting these bots to pay for API access might just be the cream of the retard crop
[ Parent | on HN ]