Why?
Maybe this is a dumb question, but why wouldn't an agent "keep the conversation going", like I do when interacting with an LLM through a web page? (I understand how it's impractical for long-running tasks where the agent has to wait days for the next input, but assume that's not the majority of use cases)
For OpenAI, it seems like you can't prolong the caching duration for money. Duration is longer during off-peak hours for in-memory caching and up to 24 hours for extended prompt caching. https://developers.openai.com/api/docs/guides/prompt-caching
For DeepSeek, caching duration of at least 12 hours (and likely longer) have been observed. Cache writes are free. https://zhuanlan.zhihu.com/p/2035737726952194774
Technically it does retrieve the entire history and reevaulate it since the LLM is stateless. Just more ergonomic for the developer.
And prompt caching helps cut the costs down when a conversation drags on.
This is also the reason providers like Anthropic scored lower because while Opus 4.7 is close to 90%, Opus 4.5 is 45%
OpenRouter is also a good place to find free LLM access with a catch: You should expect that any inputs and outputs are going into someone's training database. Clearly anyone who can pay should be using paid models with privacy protections, but the free models have been great for learning and experimenting. Especially for younger people learning API programming and LLMs who may not have access to a credit card or funds.
True enough, in theory; but what exactly are you imagining would be a useful-enough signal in the OpenRouter request+response stream, that any company would want their data as training material?
Even a single OpenRouter-API-key-identified subscriber's traffic, may consist of an mixture of traffic from multiple different sessions, under potentially multiple different end-users. (Where, if the subscriber is doing security correctly, then their OpenRouter key lives on a gateway rather than in a frontend app; and so the only IP address / UA / etc OpenRouter sees is that of the gateway itself.)
And the traffic stream may also invoke multiple models, and provide multiple different system prompts for those models; which, while marked in the traffic (i.e. conveyed as part of each request), makes the resulting data much less useful in aggregate, than if it were all training data for one model with one system prompt.
Plus, there are no RLHF signals in OpenRouter data. Even if OpenRouter wanted to build a general model-neutral framework for collecting RLHF-type data, it can't force subscriber apps to do the UI-level stuff necessary to collect it (i.e. the things ChatGPT/Claude do, with "thumbs-down" buttons, A/B tested responses, etc.) Analysis would have to rely on pure transcript-level user sentiment extraction.
https://x.com/OpenRouter/status/2041193329270878707
> Privacy:
>
> Private I/O logging and the 1% data sharing discount are separate settings. You control each independently.
>
> Input & Output Logging stores prompts and completions for your use only and makes them visible in your logs. OpenRouter does not access this data. You can configure it in your observability settings.
>
> As always, more in the docs!
https://openrouter.ai/docs/guides/features/input-output-logg...https://openrouter.ai/docs/guides/features/broadcast/overvie...
Isn't this a treasure trove for any model distillation effort?
The majority of RLHF data doesn't need this. The majority is software development and/or tool calling where the agent gets a signal back as to if it succeeded (eg compilation errors, test errors). It's true that end-of-trajectory signals (eg, did this task do what you wanted) are even more useful but even partial signals are great for RL training.
"We do not sell users’ data to third parties."
https://www.anthropic.com/news/updates-to-our-consumer-terms
I.e. nothing about this statement prevents Anthropic from running ads within Claude, as long as they run the ad-placement auctions themselves, and so aren't leaking any of the data they're using to decide which placements are relevant to which users+sessions. (This is the same thing Google does for SERP ad auctions.)
But actually, and perhaps more interestingly, nothing about this statement prevents Anthropic from building a Google AdSense competitor either. Other sites (or mobile apps, etc) could plop in an Anthropic ad iframe; and it'd be Anthropic's knowledge of your interactions with Claude that would drive what ads would show up in that iframe. The embedding site doesn't know what ads the users are seeing, so that's still not "selling users' data to third parties", per se.
Clearly, anyone who needs privacy should be using models with privacy protections. Some people build open source and the models will get the code anyway.
OpenRouter explicitly lets you filter by zero-data-retention providers: https://openrouter.ai/models?zdr=true
What's the value proposition for the typical AWS startup to go with openrouter, if Amazon offers similar rates with direct integration into all their other offerings?
The only reason OpenRouter can exist at the moment is because we are in the wild-west phase of this technology, and lots of people and companies are exploring. In 5 years they will have to have transformed their business fundamentally, or go the way of the dinosaurs.
Established restaurants didn't need DoorDash because they were already on everyone's speed dial. But new or small restaurants couldn't afford to advertise or maintain a team of delivery people. DoorDash created a two-sided marketplace that made it a lot easier for new entrants to bootstrap. Today even the established restaurants have to pay them their tithe because hungry people have learned to start with the DoorDash app. A bit of a prisoner's dilemma.
If OpenRouter plays its cards right and gets very lucky, a large number of people will configure their hungry LLM clients to start with OpenRouter, and then LLM providers will have to join the marketplace or else miss out on all those customers.
People could email cat photos and resumes. But Facebook and LinkedIn are where everyone already is, so that's what they use instead.
Everything has a cost of some sort. It's just who you're going to pay and what the currency is.
What if Fireworks stops offering your preferred model?
Anthropic: https://support.claude.com/en/articles/8977456-how-do-i-pay-... - you can pre-pay and get a hard cutoff.
OpenAI: https://community.openai.com/t/how-to-set-billing-limits-and... - last time I looked OpenAI had a soft but not hard limit, I guess they fixed that last year.
I remember bugging them both about this last year, I need to update my mental model!
Deepseek has a prepaid model. (Pretty impressive, what fits into 10 Dollar)
> Billing data processing times can be delayed in AI Studio, up to around 10 minutes. You may experience overages beyond your project cap if billing data hasn't processed before more charges are accrued.
https://ai.google.dev/gemini-api/docs/billing#project-spend-...
That's a soft cap, not a hard cap
Looks like Vercel even have their own leaderboard: https://vercel.com/ai-gateway/leaderboards/models
Surprising that they have Opus 4.8 and 4.6 listed on the leaderboard but not Opus 4.7.
Other features I've just noticed: - configurable prompt injection protection using OWASP regex (https://cheatsheetseries.owasp.org/cheatsheets/LLM_Prompt_In...) - configurable PIM protection for outbound prompts - input/output logging - "JSON healing" to auto-correct minor hallucinations
Lots of other stuff too. The business model seems pretty simple and the value-add features don't look particularly expensive or difficult to copy.
Free? They take the same 5% fee as OpenRouter does.
https://developers.cloudflare.com/ai-gateway/features/unifie...
They don't list themselves on https://openrouter.ai/providers
coffee farmers -> middle man -> you
Check out Kagi Ultimate.
For personal use, yes. The all-in pricing model encourages experimentation. And the privacy pitch seems tighter.
Maybe someday the VM I run agents in will have a dedicated GPU so that I can stop using APIs altogether. One can dream...
The difference between GLM 5.1, ChatGPT 5.4 and Opus don't really matter to me when I'm asking/talking, but programming is a different beast that needs a harness.
I’d prefer something that consolidates billing, but still lets me use providers' APIs directly (or via some "raw HTTP" proxy). There are plenty of unified API gateways, but I haven’t seen one that is just billing/auth in front of the native provider APIs.
As such, I only would use open router for things I don’t care if someone sees (or don’t have a contractual obligation to protect) and run models locally or on cloud gpus if I’m not using one of the big two.
(If they were paying me they got a bad deal, since I called out the flaws in their leaderboard approach half way through my post.)
Openrouter is very nice since it puts a barrier between you and those suppliers that were supposed to be like utilities. I got the feeling that if OpenAI was left alone they would be nice as a telco.
OpenAI and Anthropic have already done this.
Mandated BYOK will sink OpenRouter.
If you're doing any kind of production AI work you'll end up with outages caused by calling a single provider, OpenRouter seamlessly switching between providers is a godsend for uptime.
But even more than that there's meaningful cost+speed differences.
Here's Sonnet 4.6 being served direct, via Amazon and via Google
(spoiler: Google was both fastest and cheapest)
1. By far the lowest friction way to support and try out all the models.
2. They offer billing caps! Most model providers still don't do this [EDIT: maybe they do, see reply comment], but if you're going to run anything in public it's very useful to have hard limits so it doesn't cost you $1m overnight because someone started abusing it.
3. Their rankings are one of the more interesting signals for which models are popular, despite their flaws (most OpenAI and Anthropic users don't go via OpenRouter, it's currently not possible to tell the difference between many users switching v.s. one "whale" changing their preferred model)
Given how API costs are becoming meaningful for a lot of companies now, having a provider like OpenRouter to help measure your spend and easily experiment with and switch providers feels like a valuable service.