I think the big issue isn't so much a technology thing, I mean that will improve for a long while yet, but it is an economic one. My whole concern over the rapid expansion of LLM's is the massive build out on a technology that hasn't found its feet in a big enough range of markets yet that are willing to pay top dollar. Yes, world models for protein discover is very cool stuff and I kind of hate it gets lumped in with these other companies efforts because it has a very clear path forward that doesn't rely on massive IPO's just to keep the lights on.

This stuff is here to stay but I'm not sure how many of the current front runners will be able to stay solvent if they cannot turn these things in to massive money spinners. Revenue is fine-ish but spending is out of control. I see the debt in hundred of billions of dollars and start to wonder "Who is going to pay for this?" and "Will the people be willing to pay that much?". It just all feels forced rather than organic growth.

This is why I think Google may end up being one of the leaders in this field. They have their custom TPU's that seem to be fairly efficient at these tasks, they are slowly but surely improving their training and inference tech using their massive data set and most importantly, other parts of the business can subsidize this stuff for a decade if needed until it is genuinely profitable.

I am not against the industry but I do worry that many are rushing in with no means of genuine sustainability other than jump out for a golden parachute and let someone else clean up the mess.

I do hope I am wrong.