News for hackers. Stuff that matters.
Re:State of AI: An Empirical 100T Token Study with OpenRouter by lukev on Thursday December 04, 2025 @06:50PM (#46154960)
Super interesting data.

I do question this finding:

> the small model category as a whole is seeing its share of usage decline.

It's important to remember that this data is from OpenRouter... a API service. Small models are exactly those that can be self-hosted.

It could be the case that total small model usage has actually grown, but people are self-hosting rather than using an API. OpenRouter would not be in a position to determine this.

Re:State of AI: An Empirical 100T Token Study with OpenRouter by maikakz on Thursday December 04, 2025 @07:07PM (#46155116)
Thank you & totally agree! The findings are purely observational through OpenRouter’s lens, so they naturally reflect usage on the platform, not the entire ecosystem.
Re:State of AI: An Empirical 100T Token Study with OpenRouter by mzl on Friday December 05, 2025 @01:33AM (#46157438)
While it is possible to self-host small models, it is not easy to host them with high speeds. Many small-model use-cases are for large batches of work (processing large amounts of documents, agentic workflows, ...), and then using a provider that has high tps numbers would be motivated.

Still, I agree that self-hosting is probably a part of the decrease.

Re:State of AI: An Empirical 100T Token Study with OpenRouter by YetAnotherNick on Friday December 05, 2025 @03:00AM (#46157856)
The bigger issue is that they count small based on fixed number of parameters, and not the active parameter for MoE, didn't account for any hardware improvements etc. If they counted small based on the price or computational cost, I think they would have seen increase in small models.
Re:State of AI: An Empirical 100T Token Study with OpenRouter by Alex-Programs on Friday December 05, 2025 @02:57PM (#46166464)
I think using total parameters is fair, it correlates well with the RAM prerequisites to run it. Otherwise Kimi K2 would be "small" despite being a trillion parameters!
Re:State of AI: An Empirical 100T Token Study with OpenRouter by YetAnotherNick on Sunday December 07, 2025 @12:03PM (#46183148)
VRAM doesn't matter if you are using API. Price and performance is what matters.
Re:State of AI: An Empirical 100T Token Study with OpenRouter by veunes on Friday December 05, 2025 @07:36AM (#46160470)
Yeah, using an API aggregator to run a 7B model is economically strange if you have even a consumer GPU. OpenRouter captures the cream of complex requests (Claude 3.5, o1) that you can't run at home. But even for local hosting, medium models are starting to displace small ones because quantization lets you run them on accessible hardware, and the quality boost there is massive. So the "Medium is the new Small" trend likely holds true for the self-hosted segment as well.