Re:Perplexity is using stealth, undeclared crawlers to evade...
by jabroni_salad on Monday August 04, 2025 @03:47PM
(#44790593)
If it was just one human requesting one summary of the page nobody would ever notice. The typical watermark for junk traffic is pretty high as it was.
I have a dinky little txt site on my email domain. There is nothing of value on it, and the content changes less than once a year. So why are AI scrapers hitting it to the tune of dozens of GB per month?
I have a dinky little txt site on my email domain. There is nothing of value on it, and the content changes less than once a year. So why are AI scrapers hitting it to the tune of dozens of GB per month?