Re:State of AI: An Empirical 100T Token Study with OpenRouter
by mzl on Friday December 05, 2025 @01:33AM
(#46157438)
While it is possible to self-host small models, it is not easy to host them with high speeds. Many small-model use-cases are for large batches of work (processing large amounts of documents, agentic workflows, ...), and then using a provider that has high tps numbers would be motivated.
Still, I agree that self-hosting is probably a part of the decrease.
Still, I agree that self-hosting is probably a part of the decrease.