News for hackers. Stuff that matters.

A comment on Perplexity is using stealth, undeclared crawlers to evade no-crawl directives. Parent comment

Re:Perplexity is using stealth, undeclared crawlers to evade... by rjbwork on Monday August 04, 2025 @04:51PM (#44791193)
How is it wrong to send my personal shopper? How is it wrong to have an agent act directly on my behalf?

It's like saying a web browser that is customized in any way is wrong. If one configures their browser to eagerly load links so that their next click is instant, is that now wrong?

[ Parent | on HN ]
Re:Perplexity is using stealth, undeclared crawlers to evade... by ghurtado on Monday August 04, 2025 @05:17PM (#44791464)
Here's a good rule of thumb: if you have to do it without other people knowing, because otherwise they wouldn't let you do it: chances are it's a bad thing to do.
[ Parent | on HN ]
Re:Perplexity is using stealth, undeclared crawlers to evade... by _proofs on Monday August 04, 2025 @09:14PM (#44793268)
if you send your personal shopper to a store, and the business is... closed for business, or refusing you entry, and you just... go in anyway.

that's called breaking and entering, and generally frowned upon -- by-passing the "closed sign".

[ Parent | on HN ]