News for hackers. Stuff that matters.
Re:Perplexity is using stealth, undeclared crawlers to evade... by lukeschlather on Monday August 04, 2025 @10:44AM (#44786515)
You probably need a computer that costs $250,000 or more to run the kind of LLM that Perplexity uses, but with batching it costs pennies to have the same LLM fetch a page for you, summarize the content, and tell you what is on it. And the power usage similarly, running the LLM for a single user will cost you a huge amount of money relative to the power it takes in a cloud environment with many users.

Perplexity's "web crawler" is mostly operating like this on behalf of users, so they don't need a massively expensive computer to run an LLM.

Re:Perplexity is using stealth, undeclared crawlers to evade... by sterlind on Monday August 04, 2025 @07:06PM (#44792353)
does Perplexity store crawled pages for training?