Re:First Impressions with GPT-4V(ision)
by dmazzoni on Wednesday September 27, 2023 @03:51PM
(#37680288)
First of all, you're off by an order of magnitude.
Second, I don't think it will be that long. There are already LLMs as good as GPT-3 running on average laptops and even phones.
In the next couple of years, you'll see:
- Ordinary PCs, tablets, and phones with dedicated AI chips, like TPUs - they'll be more tuned specifically for LLMs
- Mathematical and algorithmic optimizations will make existing LLMs faster on the same hardware
- Newer generations of LLMs will get even more useful with fewer parameters
The combination of all of these means that it's not at all unreasonable to expect that today's top-of-the-line LLM will be running locally on your device within just a couple of years.
Of course, LLMs in the cloud will advance even further, so there will always be a tradeoff, and there will always be demand for cloud AI, depending on the application.
Re:First Impressions with GPT-4V(ision)
by fomine3 on Wednesday September 27, 2023 @09:48PM
(#37684184)
I don't know. RAM is $$ and currently usable LLMs needs huge RAM with higher bandwidth. I don't see any story that it will be solved with future AI chips. Do you know anything?
Second, I don't think it will be that long. There are already LLMs as good as GPT-3 running on average laptops and even phones.
In the next couple of years, you'll see:
- Ordinary PCs, tablets, and phones with dedicated AI chips, like TPUs - they'll be more tuned specifically for LLMs
- Mathematical and algorithmic optimizations will make existing LLMs faster on the same hardware
- Newer generations of LLMs will get even more useful with fewer parameters
The combination of all of these means that it's not at all unreasonable to expect that today's top-of-the-line LLM will be running locally on your device within just a couple of years.
Of course, LLMs in the cloud will advance even further, so there will always be a tradeoff, and there will always be demand for cloud AI, depending on the application.