I think we are not far off this with Pi, Muse Glimmer, Qwen, Gemma etc., but sooner or later we will all have to work out how to put money behind it.
The main problem is that the open source movement has trouble even raising the money to properly pay linchpin individuals like Harlan Stenn (NTP).
Even Omarchy, which is uncomfortably effective at raising money for what amounts to a barely differentiated Linux distribution, is not raising LLM-training money; it’s barely even raising the kind of money it would take to post-train Qwen, I think.
We're in the 80s era in the OS analogy.
Gotta wait until there's a race to the bottom, prices drop and a bunch of these projects to build an AI model turn out to lose money. Then people will realize it's financially more sensible to pitch in and collaborate on one good model instead of building their own.
FWIW it is worth noting that the GP's suggestion is more in line with Microsoft's own stated belief in the future of "unmetered AI".
At the moment, Microsoft is arguably meeting that claim with their support for and encouragement of Nvidia Nemotron Bolt, which is an actually pretty good, quite interesting, unusually-trained-and-sized-to-fit MoE model that was designed to go with the launch of the Nvidia/Microsoft RTX Spark machines.
(Good Sam Witteveen video about it: https://www.youtube.com/watch?v=WJpeW9lCSUM )
But it would be good to see Microsoft themselves distilling an open weights model.
Like any vendor with a profit motive they do have to decide the obvious, though: how do you make money from it, when right now it could cost several hundred million dollars to train (even if you adjust the training of a larger proprietary model so you can distill the smaller one from it)?
There is, essentially, no way to keep an open weights model from being copied/distilled/extracted, and IMO what that tends to suggest is that open weights models will essentially gravitate to sort of uncontroversial, "standard library" tasks.
So I think it also means that Microsoft's approach on open weights will ultimately be the same as Apple's: they will focus their efforts on LLMs that meaningfully add function to their OS and can be decapitated to make similarly useful decision models.
Put another way: given the geopolitical angle of the last few weeks, I think it is possible that the open weights coding models we have now may be among the best we will ever get, until the nearly inevitable GPU/tensor hardware glut arrives.