I was more skeptical of openrouter a year ago. Not hard to implement multiple models, but after getting burned multiple times (eg Google in particular) I see openrouter as serving a few key functions: (1) aggregating demand across neoclouds for performance etc (2) providing a great developer experience (3) easily switching models and normalizing the idiosyncratic nature of each. There are so many (good) model providers today this is more compelling now with 10+ solid model companies than a year ago when I thought it might be just 3.

So I have shifted load away from calling some models to openrouter.

This is really interesting, because I'm actively building an open source SaaS router product. We're starting in the FAL area rather than the OpenRouter area (much higher margin), but they eventually intersect.

My thinking is businesses will want the ability to fire their provider and bring things on-prem if they want (but that they'd rather stick with a good provider as long as they're happy). Not so much that they'll do it, more that they'll want the option.

Flexibility isn't super important now as most routers are fungible, but as we move into an era dominated by open source models and fine tunes, that becomes a major impediment to switching. So building this now seems especially important. Especially if we guarantee data and model exports to customers.

A customer with fine tunes of open source models won't find it simple and easy to migrate. But we can build this escape hatch now before that story has a chance to evolve.

And of course there's a lot of add-ons that can be sprinkled on top later: handling customer data/outputs (videos, thumbnails, etc.), spinning up domains for products (eg. Three.js video game LLM outputs), etc. Vercel/Itch.io as a service, Vimeo as a service, media catalogue search as a service, etc. It'd be nice if this was all open source and portable.

Is anyone here interested in a closed beta of a Rust-based "Open" OpenRouter, Fal, (and eventually RunPod and Featherless)? We host it for you, but the entire stack is OSS and you can bring it on-prem if you want. I'd be happy to send invites. My email is "echelon at gmail".

We also do BYOK forwarding, so it's super simple to switch without even paying us. We even support router keys for double routing (OpenRouter and FAL keys in addition to OpenAI, Google, etc.)

Might need to rethink our Stripe payment processing. They have our customer book.

I would not be suprised to see Cloudflare launch a competitor in the next 6 mo. They've done similar before, and it fits their "middle man/service provider" vibe well.

OpenRouter is good because of the market ecosystem play. I see this fragmenting similar to game distribution -- a handful of 3rd-party disti, 1 first-party disti, and some exclusivity/lockup/limited support outside of first party.

I am very interested to see how Stripe develops this. I would be happy to see a less-clunky top-up solution, for example, perhaps with micropayments. Not sure what else they are planning.

Cloudflare AI Gateway launched in beta in 2023, and went GA in 2024.

Recent overview: https://blog.cloudflare.com/ai-platform/

Part of the issue -- cloudlfare is an inference provider for models (eg glm-4.7-flash) on openrouter. Very very hard to get partners to signup if you're going to directly compete with them.

It's pretty bad though compared to openrouter tbh. I expected it to have some inbuilt cloudflare access access control, but it doesn't have that for improving security either within the team Easier to just use openrouter tbh.

Basically if you look at OpenRouter in a isolation this could be confusing, but when bundled with Stripe's integrated product solutions for the lower end, developer centric market, this makes infitite sense. https://s-1.vercel.app/posts/what-stripe-can-become-broader-...

i like ur posts. have u considered recording audio for them?

I have although I would prefer just riffing with someone every week, similar to the Exponent Podcast.

The harness is the router. I don't understand this.

You don't have to spread your payment info to 10 different places ? You pay a fee for centralized API credits

Payment processing for tokens - sounds like the thing Stripe does.

Why do you need 10? One or two fallbacks is sufficent. Maybe my workloads are vastly different than yours but I’m not a huge fan of having to switch between models. Even within a single model determinism sucks, debugging why something worked yesterday and doesn’t work today because a model switch is not fun. I can’t imagine the pain of having our services flip randomly between 10 different models.

It is changing so often that to keep the costs and quality at a pareto level you need to experiment with a lot of different providers. And the pain you are describing should be a normal process in AI. I use model pools where the tasks are optimized for multiple AI providers. For example I was experimenting with GPT Luna and it turned out that the model is good but tool shy so I had to improve the instructions. Now this model is my main model for the chat in my app. Next week it can be a different model.

It might just not be for you. I use OpenRouter because I do like being able to quickly check whether a new model works better, but mine is a “human in the loop” dev process so it won’t ruin a day of batch processing or anything.

I do like being able to try eg GLM without having to set up a new account. It’s also nice that I don’t have to top up per-model accounts. I think I have a couple accounts with $7 in API credits sitting around.

Probably not something I would do for actual business processing, where the stability is dramatically more important than tinkering with new models.

Agree with you

makes sense - https://s-1.vercel.app/posts/why-openrouter-can-be-the-next-...

Where is the guy who marks all the dupes? Day off? https://news.ycombinator.com/item?id=49323381

I both think OpenRouter is not worth 7B$ and Stripe might have made a brilliant move in making this purchase, honestly just troubling thoughts around model inference and model token as the new traded currency go through my head.

Imagine if Stripe's end game is to control which producers of tokens become visible for a price aka middleman tax for tokens. If AI is as big a change as people claim honestly AWS should have bought openrouter instead.

But either way though 7B$ is just too rich no matter what, because the middleman tax is only worth it if you have a moat which OpenRouter doesn't.

All it takes is one of the bigger players to get serious and they will have a similar platform up and working in months if not weeks. (like Vercel & Cloudflare are already doing but even they are small compared to the true behemoths)

If hyperscalers start their own open routing service and cut off openrouter completely and become price competitive, I feel like they could wipe them out.

7B$ is just too much. Without enough of a big defensible moat. Especially since you can use a router in-front of openrouter and move users piece meal.

Can anyone explain what makes OpenRouter specifically worth 7B$ over everyone else? I say that as someone who has used a lot of similar services with similar results. Openrouter is better but only marginally.

I specifically think many of the hyperscalers are a bad choice to platform on because they have their own models. You cannot build a viable platform business while directly competing with your potential customers and partners, the incentives are too out-of-whack.

Stripe/CF/Vercel are presumably not going to do foundational model development, so safe there. They are already platform plays to begin with, this strengthens that position.

I can bet AWS won't be doing any serious foundational model development, I knew folks on their AGI team now it's not even big enough to compete with the chinese labs... :3

Azure I am not sure how long they will be around doing MAI thinking models, I feel like they will pivot to smaller simpler enterprise only models. And given they already own npm and github openrouter might have made sense though I think they might buy huggingface, not that I want them to but it just feels like it.

Google IDK what google is doing exactly all my friends I knew in the AI teams are out a while ago, so maybe they are doing something really great we just don't know yet.

But I feel like it makes sense for hyperscalers since they can push their weight around a lot better than openrouter and can offer extra compute when providers are under crunch at higher prices to handle spikes. I think they are the only ones who can truly do fluid compute for GPU/AI in the short term(next couple of years).

Maybe after than we might have other big players in the space. Given the sheer scale of buildout I can almost guarantee we will see this pivot, otherwise there is too much hardware and token prices are too high.

IDK what happened to AWS, other than Amazon's cultural reputation is so bad it may be hard to hire the best eng.

Azure, GCP, I generally agree. The Neoclouds will drive prices into the basement, I think custom silicon/data moat/ease to deploy will be the hyperscaler's edge.

AWS is too busy selling entire private AWS regions on "take or pay" agreements to Anthropic, OpenAI and Meta.

Not exactly related, but I've seen people suggesting a new path for opens source funding that would be around tokens instead of cash. One might be able to pledge tokens towards a feature or fix instead of cash.

Even the suggestion of this absent an implementation hints at a future for tokens as a commodity. But it is interesting to think of it as some kind of new asset class, between commodity and currency.

I mean, we already think of tokens in the form of currency in crypto (e.g. NFT). There may be more than just similarity in the name choice when it comes to AI tokens. One could even think of things like "exchange rates", etc.

positioning for commoditization of intelligence

OpenRouter is a two-sided inference market with an established network and spinning flywheel. If Stripe can manage not killing it, it is going to print them money for a long time.

edit: Fixed the poor AM English.

OR has already established a “5% processing fee”. Right up Stripe’s alley.

(I switched to a major bank who offers much better pricing and integrates the Clover API.)

If LLM providers continue grow and become such a standard part of the tech stack of any deployment, this acquisition makes a lot of sense. It's just another payment management system for stripe.

It's fun to see someone being able to exit so well without owning any model.

The acquisition makes no sense, as most LLM providers only reason not to forbid them right now, is because they were funded by exactly the same VCs as the providers. Go check.

Of all the no moats they are the example of no moat. Its seconds switch to another router if even that is tolerated. And lets not even talk, the lack of compliance in an enterprise environment, and as they defer to providers on that...

But the acquisition makes a LOT of SENSE, if you realize that as the same VCs sit on Stripe, Anthropic, OpenAI and most important OpenRouter. Said VCs have a lot of interest in recovering their money...by pushing one of their companies to buy another one of their own, before LLMs providers financial pressures cut OpenRouter legs.

Is this stuff even legal? I will wait for Patrick Boyle video...

> And lets not even talk, the lack of compliance in an enterprise environment, and as they defer to providers on that...

A middleman that adds compliance & regulation while skimming a slice off the top for themselves: You just described credit cards - stripe's core business...

Very much the same line of business as Amazon Bedrock except OpenRouter gives more options (choice of where hosted, data retention options and so on).

>> choice of where hosted,

You dont understand Bedrock fundamental use case.

Yeah I do. You can get data locality in GovCloud, a U.S. data centre, or even EU regions.

It’s governed by your master AWS agreement (which will make your compliance officer happy), and billing is also conveniently handled by AWS.

Of course the Chinese models on it are rotten old and not cheap. But if you want to configure an app to use inference that is entirely controlled, it’s the next best thing to running local.

Most important...

- Your inputs are within the security boundary of your AWS account or AWS accounts

  Can OpenRouter say the same? - NO

- Your inputs are not shared with models creators

   Can OpenRouter say the same? - NO

- Your inputs are not used to train the next model

   Can OpenRouter say the same? - NO

- Within certain limits and certain models:

   "50.10.1. Subject to the limitations in this Section 50.10, AWS will defend you and your employees, officers, and directors against any third-party claim alleging that the Generative AI Output generated by an Indemnified Generative AI Service infringes or misappropriates that third party’s intellectual property rights, and will pay the amount of any adverse final judgment or settlement."

  Can OpenRouter say the same? - NO

> Within certain limits and certain models

For those that hadn't read yet (on https://aws.amazon.com/service-terms/):

"Certain models" means "only AWS's own trained models" (Nova etc), so not e.g. Bedrock Claude or GPT.

"Certain limits" means they don't defent if the input to the model is the IP.

How is the middleman going to add compliance if its not the provider of the LLMs? And if that was possible OpenRouter would have done it already...

You over-estimate the amount of sway investors have on Stripe. If VCs had the influence you're describing, Stripe would have gone public many years ago.

Another bottomless money pit.

7B for a wrapper.

Stripe isn’t exactly a dumb company. When someone smart does something seemingly super dumb, more likely than not you’re the one missing something, not them. It’s an opportunity to ask, “what am I missing?”, and learn from it.

I didn't say it was super dumb, it was more of a reflection on that's how much a wrapper is worth today.

> I didn't say it was super dumb, it was more of a reflection on that's how much a wrapper is worth today.

Are you sure that the value is in the wrapper and not rather in the fact that investors who invested into OpenRouter also invested into companies that can buy OpenRouter? ;-)

(see also https://news.ycombinator.com/item?id=49330668 ).

It is not just the software. Let's not forget the existing customer base, an opportunity cost to build something as mature as openrouter (even with LLMs). Perhaps it is worth it for them.

> Stripe isn’t exactly a dumb company.

I guess hardly anybody claims that. I think the suspicion is rather about whether there exist some other "puppetmasters" (e.g. venture capitalists) who pressured Stripe into buying OpenRouter.

Stripe is a wrapper too. Single API key and unified DX atop a world of divergent providers.

Adding support to yet another LLM API provider is order of magnitude easier (at least) than adding anything in finance domain

For a wrapper that collects a bunch of data. Think of this purchase like Zucks investment in ScaleAI.

No not all, it is a broader bundling strategy for Stripe - https://s-1.vercel.app/posts/what-stripe-can-become-broader-...

That's just one take on the situation. What makes up that bundle is what I'm referring to. They aren't just blindly buying companies. Why exactly is OR interesting to Stripe?

For traction...