Show HN: Route your prompts to the best LLM

Re:Show HN: Route your prompts to the best LLM by Oras on Thursday May 23, 2024 @06:32AM (#40452950)

I've been using a prompt management service (cloud) for 2 months and am pretty happy that I can define prompts and check quality outside the code, and it helps me to do the routing manually.

Having benchmarks (I assume this is ongoing?) would make it even more interesting, as I wouldn't have to manually manage the routing each time I add a new model.

You mentioned you don't have a margin right now, but how about keeping logs, the dashboard for cost, and benchmarking?

[ Parent | on HN ]

Re:Show HN: Route your prompts to the best LLM by danlenton on Thursday May 23, 2024 @06:40AM (#40452996)

Yes the benchmarks are ongoing, we continually plot the speed and cost across time in our runtime benchmarks [https://unify.ai/benchmarks], and we use this live data when plotted the quality scatter graphs [https://console.unify.ai/dashboard]. The router configurations are "self-improving" in the sense that any given router config will quickly wrap the latest models and providers under the hood. Using a router config is a way of riding the wave of models and providers, whilst just specifying your priorities for quality, speed and cost. We will have some case studies which better explain this soon!

[ Parent | on HN ]