Re:Meta rolls out AI language model LLaMA
by generalizations on Friday February 24, 2023 @03:45PM
(#34929872)
> The future interaction will AI models will most likely be through an API because the models themselves are becoming too large to fit even on the most extreme DIY NAS solutions.
And yet I spent last night running GPT-J-6B on my desktop CPU at 2 tokens / sec. People are finally starting to optimize these models, and there's a ton of optimization to go. We'll definitely be running these locally in the next few years. This model especially looks like an ideal candidate for CPU optimization, given the pairity with GPT3, and that it's within spitting distance of the size of models like GPT-J-6B.
Re:Meta rolls out AI language model LLaMA
by Hyption on Friday February 24, 2023 @03:57PM
(#34930025)
There are plenty low paid students or working students or admins in-between.
Re:Meta rolls out AI language model LLaMA
by imranq on Friday February 24, 2023 @05:37PM
(#34931155)
You can make small LLMs more performant than large ones like GPT-3 by fine-tuning on specific tasks or providing them tools to offload precise calculations:
e.g. Toolformer: https://arxiv.org/abs/2302.04761
which uses APIs and functions to improve GPT-J beyond GPT-3 for various tasks
Re:Meta rolls out AI language model LLaMA
by 1letterunixname on Friday February 24, 2023 @05:50PM
(#34931303)
How dare you bring knowledge in here! Some people at work would be offended if they couldn't buy their A100 or DGX toys, or had to scale back their custom ASIC and systems R&D.
The career risk isn't worth it especially when tech is deployed client- and network-side to detect just such exfil attempts. The average of network, security, and client management staff tend to be PEs (SREs) who can code, some have PhDs, and are the cream of what was previously organized as "corporate IT" world. So I fail to see any incentive to throw away their career and reputation by giving away IP for $0.
There's a metric s*ton of optimized hardware to generate models. And I have my doubts if Sama at OpenAI, even with 10 gigabucks from Microsoft, can sustain growth, organizational culture, and long-term investment at the scale others are bringing online with less trouble and more experience.
The future interaction will AI models will most likely be through an API because the models themselves are becoming too large to fit even on the most extreme DIY NAS solutions.
TL;DR: it's not happening.