The bigger threat is if their models "stall", while a new up-start discovers an even better model/training method.

What _could_ prevent this from happening is the lack of available data today - everybody and their dog is trying to keep crawlers off, or make sure their data is no longer "safe"/"easy" to be used to train with.

They can also buy out the startup or match the development by hiring more people. Their comp packages are very competitive.

But, if model development stalls, and everyone else is stalled as well, then what happens to turn the current wildly-unprofitable industry into something that "it makes sense to keep spending billions" on?

I suspect if model development stalls we may start to see more incremental releases to models, perhaps with specific fixes or improvements, updates to a certain cutoff date, etc. So less fanfare, but still some progress. Worth spending billions on? Probably not, but the next best avenue would be to continue developing deeper and deeper LLM integrations to stay relevant and in the news.

The new OpenAI browser integration would be an example. Mostly the same model, but with a whole new channel of potential customers and lock in.

If model development stalls, then the open weight free models will eventually totally catch up. The model itself will become a complete commodity.

It very well might. The ones with most smooth integrations and applications will win.

This can go either way. For databases open source integration tools prevailed, the commercial activity left hosting those tools.

But enterprise software integration that might end up mostly proprietary.

Because they’re not that wildly unprofitable. Yes, obviously the companies spend a ton of money on training, but several have said that each model is independently “profitable” - the income from selling access to the model has overcome the costs of training it. It’s just that revenues haven’t overcome the cost of training the next one, which gets bigger every time.

> the income from selling access to the model has overcome the costs of training it.

Citation needed. This is completely untrue AFAIK. They've claimed that inference is profitable, but not that they are making a profit when training costs are included.

I've also seen Open AI and Anthropic say it's pretty close at least. I'll try to follow up with a source.