Re:Show HN: AI Proxy with Support for OpenAI, Anthropic, LLa...
by alanaan on Monday November 20, 2023 @12:40PM
(#38351279)
how does this differ from a model “router” (e.g. withmartian.com)?
[ Parent
| on HN ]
Re:Show HN: AI Proxy with Support for OpenAI, Anthropic, LLa...
by ankrgyl on Monday November 20, 2023 @12:48PM
(#38351423)
Model routers are a "semantic" abstraction -- they analyze the contents of a prompt and automatically determine which provider to use. This proxy is (currently) much simpler: you pick the provider you want and we add a bunch of nice features like caching and low-latency on top.
We will likely work with or build some routing capabilities in the future!