News for hackers. Stuff that matters.

AMeckes

Show HN: Any-LLM chat demo – switch between ChatGPT, Claude, Ollama, in one chat

https://github.com/mozilla-ai/any-llm/tree/main/demos/chat

any-llm is a library that provides unified access to multiple LLM providers. switching between providers like OpenAI, Anthropic, Google, Mistral, and even Ollama is just a string change. This demo makes it easy to look at and compare responses from different models. Interesting to see how ChatGPT, Claude, Gemini, and local models each reason through the same problems. Let us know what you think!

( Read More... | no comments | 6 points | on HN )
We built mcpd to make MCP servers easier to work with. It's a daemon that runs MCP servers as subprocesses and exposes a unified HTTP API. Write your config once in .mcpd.toml and it works everywhere. Let us know what you think!
Both approaches work well for standard text completion. Issues tend to be around edge cases like streaming behavior, timeout handling, or new features rolling out.

You're absolutely right that any router reimplements interfaces for normalization. The difference is what layer we reimplement at. We use SDKs where available for HTTP/auth/retries and reimplement normalization.

Bottom line is we both reimplement interfaces, just at different layers. Our bet on SDKs is mostly about maintenance preferences, not some fundamental flaw in LiteLLM's approach.

I didn't even need to click the link to know what this comic was. LiteLLM is great, we just needed something slightly different for our use case.
Python only for now. Most providers have official TypeScript SDKs though, so the same approach (wrapping official SDKs) would work well in TS too.
That's true. We traded API compatibility work for SDK compatibility work. Our bet is that providers are better at maintaining their own SDKs than we are at reimplementing their APIs. SDKs break less often and more predictably than APIs, plus we get provider-implemented features (retries, auth refresh, etc) "for free." Not zero maintenance, but definitely less. We use this in production at Mozilla.ai, so it'll stay actively maintained.
To clarify: any-llm is just a Python library you import, not a service to run. When I said "put it behind a proxy," I meant your app (which imports any-llm) can run behind a normal proxy setup.

You're right that archgw handles routing at the infrastructure level, which is perfect for centralized control. any-llm simply gives you the option to handle routing in your application code when that makes sense (For example, premium users get Opus-4). We leave the architectural choice to you, whether that's adding a proxy, keeping routing in your app, or using both, or just using any-llm directly.

Good points! any-llm handles the LLM routing, but you can still put it behind your own proxy for centralized control. We just don't force that architectural decision on you. Think of it as composable: use any-llm for provider switching, add nginx/envoy/whatever for rate limiting if you need it.

Show HN: Any-LLM – Lightweight router to access any LLM Provider

https://github.com/mozilla-ai/any-llm

We built any-llm because we needed a lightweight router for LLM providers with minimal overhead. Switching between models is just a string change : update "openai/gpt-4" to "anthropic/claude-3" and you're done.It uses official provider SDKs when available, which helps since providers handle their own compatibility updates. No proxy or gateway service needed either, so getting started is pretty...

( Read More... | 68 comments | 56 points | on HN )