'Better' has many meanings. Ultimately LLM performance depends not only on processor performance, but also on data. One could imagine real-time self-training, where the real-time availability of (allegedly anonymized) data from other users leads to a stable advantage over local LLM.
Another issue is deployment. Imagine this: You are a software developer using an LLM in your commercial product. You have two/three choices.
A) Call a cloud API using a simple cross-platform HTTP call. This will reliably work. While at it, you may gain useful data on your customer.
B) Include a local LLM, and deal with all the lovely deployment issues, especially when the guys running a 8th gen Intel laptop with an iGPU and 4GB memory call and ask why they can't run your product.
Perhaps there will be a third option: C) Call a local API provided by the OS, which may or may not call a cloud API in the background.
One can see how even if the performance is totally equivalent, Cloud API may end up with an advantage.
IMHO, this is not a desirable future, but to avoid it we need to start by cutting down on the triumphalism.
>Call a cloud API using a simple cross-platform HTTP call. This will reliably work. While at it, you may gain useful data on your customers
In the not too distant future, I expect most applications to be replaced by LLM assistants. So the above scenario won't really happen. You will interact with one LLM, maybe it's a local flavor or maybe it's a cloud flavor. The LLM will then call other services for you. So I don't expect services to call LLMs but I do expect LLMs to call other services.
Yes, we are getting a little ahead of ourselves.
>In the not too distant future, I expect most applications to be replaced by LLM assistants. So the above scenario won't really happen.
Regardless we have the question of how to deploy the local LLM. If the frontend will be the OS, than we can be reliably sure that Microsoft and Google will have motivation to use their cloud hosted AI and you may have to fight the OS to get it to install your model. We may need regulation to ensure local LLMs are possible.
The same as we do any other software?
> If the frontend will be the OS
Why would the frontend be the OS? The OS isn't the frontend to my database. The OS isn't my IDE. The OS isn't the games I play. The OS isn't even my browser.
All these are Userspace programs.
I answered this above.
If the cloud versions are significantly better (like they are now) than local LLMs, then cloud will continue to be the way. If local LLMs reach 95% of what cloud versions can do, then I think local ones might win out because the cost will be smaller, it will be faster (in latency), and it will have more privacy.
In 3-4 years? I'm willing to bet that local LLMs will have a sizable market.
I basically expect the neural engine inside an Apple Silicon M7 to be 80% of the SoC - instead of 10% like it is today. We're going to be buying NPUs (Neural Processor Unit) with a CPU and GPU attached to it. Right now, we're buying a CPU with a GPU and NPU attached to it.
I don't expect giant corporations to control LLMs. I expect smaller companies to be able to compete and that techniques for training and deploying LLMs will eventually look similar to how software is built today - where anyone can train and deploy LLMs.