News for hackers. Stuff that matters.
Re:Show HN: I've built a locally running Perplexity clone by fnetisma on Wednesday April 03, 2024 @07:48PM (#39924830)
This is really neat! I have questions:

“Needs tool usage” and “found the answer” blocks in your infra, how are these decisions made?

Looking at the demo, it takes a little time to return results, from the search, vector storage and vector db retrieval, which step takes the most time?

Re:Show HN: I've built a locally running Perplexity clone by nilsherzig on Wednesday April 03, 2024 @07:56PM (#39924890)
Thanks :)

Die LLM makes these decisions on its own. If it writes a message which contains a tool call (Action: Web search Action Input: weight of a llama) the matching function will be executed and the response returned to the LLM. It's basically chatting with the tool.

You can toggle the log viewer on the top right, to get more detail on what it's doing and what is taking time. Timing depends on multiple things: - the size of the top n articles (generating embeddings for them takes some time) - the amount of matching vector DB responses (reading them takes some time)

Re:Show HN: I've built a locally running Perplexity clone by dcreater on Thursday April 04, 2024 @01:53AM (#39927000)
> Die LLM

You mean the? The German is bleeding through haha

Re:Show HN: I've built a locally running Perplexity clone by rzzzt on Thursday April 04, 2024 @02:13AM (#39927090)
Wolfenstein 3D did it first! And then The Simpsons as well.