> If making money on inference alone was possible, there would be a dozen different smaller providers who'd be taking the open weights models and offering that as service.

There are! Look through the provider list for some open model on https://openrouter.ai . For instance, DeepSeek 3.1 has a dozen providers. It would not make any sense to offer those below cost because you have neither moat nor branding.