Re:Amazon vs. Perplexity – U.S. Court of Appeals for the Nin...
by algoth1 on Monday September 14, 2026 @05:17PM
(#49704175)
TLDR: Perplexity agents allegedly ignored Amazon's robots.txt
[ Parent
| on HN ]
Re:Amazon vs. Perplexity – U.S. Court of Appeals for the Nin...
by Zambyte on Monday September 14, 2026 @05:26PM
(#49704309)
robots.txt is a suggestion, not a rule. It's a service to crawlers to help them avoid wasting time. Using robots.txt as a security measure is like trying to stop a foreign invader with a "road closed" sign.
Edit: this is obviously assuming they literally meant robots.txt, but from a qyick skim of the site, it doesn't look like that was mentioned at least. I guess you meant it metaphorically :P
Re:Amazon vs. Perplexity – U.S. Court of Appeals for the Nin...
by shermantanktop on Monday September 14, 2026 @08:28PM
(#49706131)
> robots.txt is a service to crawlers to help them avoid wasting time
I doubt anyone edits robots.txt with a warm fuzzy feeling about how much they are helping crawlers be efficient.