News for hackers. Stuff that matters.

A comment on Show HN: Route your prompts to the best LLM.

Re:Show HN: Route your prompts to the best LLM by Animats on Thursday May 23, 2024 @06:14AM (#40452807)
That makes a lot of sense for anything that's public-facing and free.

If the user is having a dialog with the system, they start out with a trivial question, and are routed to a simple LLM, are they stuck there? Or can the conversation be transitioned to something more powerful?

Something like this will be needed to direct traffic as systems specialize more.

[ Parent | on HN ]
Re:Show HN: Route your prompts to the best LLM by danlenton on Thursday May 23, 2024 @06:21AM (#40452863)
Great question! Nope they're not stuck with the simple LLM after the first question. Chatbots simply aggregate all prior conversation history in the input prompt, and make a new LLM call. The router treats each LLM call seperately, and so a single conversation can occur across many models and providers.
[ Parent | on HN ]