> Firstly, the capex is currently too high for all but the few.

> This is a rather obvious statement,

Nobody is saying companies have to make LLMs themselves.

SASS is a thing.

By SAAS I assume you mean public LLMs, the problem is the hand-wringing occurring over intellectual property leaking from the company. Companies are actually writing policies banning their use.

In regards to Private LLMs, the situation has become disappointing in the 6 months.

I can only think of Mistral as being a genuine vendor.

But given the limitations in context window size, fine tuning is still necessary, and even that requires capex that I rarely see.

But my comment comes from the fact that I heard from several sources, smart people say "we tried language models at work and it failed".

However in my discussion with them, they have no concept of the size of the datacentres used by the webscalers.

It's not clear to me that fine-tuning is even capex. If you fine tune new models regularly, that's opex. If you mean literally just the GPUs, you would presumably just rent them right? (Either from cloud providers for small runs or the likes of sfcompute for large runs) Or do you imagine 24/7 training?

This is a good reminder that every org is different. However some companies like Microsoft are aggressively pushing AI tools internally, to a degree that is almost cringe.

But this is often a mixture of these two things.

The tools are often cringe because the capex was laughable. E.g. one solution, the trial was done using public LLMs and then they switched over to an internally built LLM which is terrible.

Or, secondly, the process is often cringe because the corporate aims are laughable.

I've had an argument with a manager making a multi-million dollar investment in a zero coding solution that we ended up throwing in the bin years later.

They argued that they are going with this bad product because "they don't want to have to manage a team of developers".

They responded "this product costs millions of dollars, how dare you?"

How dare me indeed...

They promptly left the company but it took 5 years before it was finally canned, and plenty of people wasted 5 years of their career on a dead-end product.

I don't want to shill for LLMs-for-devs, but I think this is excellent corporate strategy by Microsoft. They are dog-fooding LLMs-for-devs. In a sense, this is R&D using real world tests. It is a product manager's dream.

The Google web-based office productivity suite is similar. I heard a rumor that at some point Google senior mgmt said that nearly all employees (excluding accounting) must use Google Docs. I am sure that they fixed a huge number of bugs plus added missing/blocking feature, which made the product much more competitive vs MSFT Office. Fifteen years ago, Google Docs was a curiosity -- an experiment for just how complex web apps could become. Today, Google Docs is the premiere choice for new small businesses. It is cheaper than MSFT Office, and "good enough".

Google docs has gotten a little better in that time, but it's honestly surprisingly unchanged. I think what really changed is that we all stopped wanting to layout docs for printing and became happier with the simpler feature set (along with collaboration and distribution).