Re:Show HN: Route your prompts to the best LLM
by swyx on Thursday May 23, 2024 @12:40AM
(#40450539)
yeah this is very nice, open source Martian. i never really understood the value of routing all the time. you want stability and predictability in models. and models have huge brand value. you're never going to, through routing, construct a "super" model that people want more than one or a few really good brand name models.
[ Parent
| on HN ]
Re:Show HN: Route your prompts to the best LLM
by danlenton on Thursday May 23, 2024 @04:47AM
(#40452198)
Yeah I agree, routing all the time is overkill for some applications, however for others it really is necessary in order to reach the necessary speed and performance for users of your real-time app.
For those who don't want to always route, another core benefit of our platform is simple custom benchmarking on your task across all existing providers: https://youtu.be/PO4r6ek8U6M
If you then just want to use that provider rather than a router config, then that's fine!