News for hackers. Stuff that matters.

A comment on Perplexity is using stealth, undeclared crawlers to evade no-crawl directives. Parent comment

Re:Perplexity is using stealth, undeclared crawlers to evade... by BrawnyBadger53 on Tuesday August 05, 2025 @01:07AM (#44794515)
Anything but expanding copyright laws. Tbh, a pay per citation with an opt in database to add your info (think music streaming style monetization) would be reasonable to me. Not that I think it's a good scheme for music but I think it's fitting for web crawling. Though it does inevitably lead to enshitification. Pick your poison I guess.
[ Parent | on HN ]
Re:Perplexity is using stealth, undeclared crawlers to evade... by wulfstan on Tuesday August 05, 2025 @03:58AM (#44795429)
The reason it works for music is because the people behind the databases have a team of lawyers that will come after you for violating copyright/performance legislation if you don’t pay your dues.

The argument that LLM outfits are using is that they are just exercising “fair use” / education rights to do an end run around copyright law. Without strengthening the rules on that I’m not sure I see how the database + team of lawyers approach would work.

But with that, sure, that’s an approach that seems to have legs in other contexts.

[ Parent | on HN ]