News for hackers. Stuff that matters.
Re:Show HN: I'm making an AI scraper called FetchFox by padolsey on Wednesday September 04, 2024 @03:19AM (#41442779)
Technically it's acting on behalf of a proactive user in Chrome so IMHO is non-"robotic". But heh tbf this was also the excuse of Perplexity where they argued they are a legitimate non-robotic-user-agent (thus don't need to respect robots.txt) because they only make requests at the time of a user query. We need a new way of understanding what it even means to be a legitimate human user-agent. The presence of AIs as client-side catalysts will only grow.

There are no comments on this comment yet.