Re:Get answers for Shell Commands from GPT3 right from your ...
by bheadmaster on Friday November 18, 2022 @03:04AM
(#33651755)
I don't like the concept of having my terminal send my commands to a remote service, so I'll pass. Hopefully an open source model comes around soon so stuff like this cam be implemented locally.
[ Parent
| on HN ]
Re:Get answers for Shell Commands from GPT3 right from your ...
by abhagsain on Friday November 18, 2022 @03:20AM
(#33651841)
That sounds great!
Re:Get answers for Shell Commands from GPT3 right from your ...
by moffkalast on Friday November 18, 2022 @06:41AM
(#33653274)
Sure, as long as you've got 350 GB of VRAM to load it.
Re:Get answers for Shell Commands from GPT3 right from your ...
by visarga on Friday November 18, 2022 @11:39AM
(#33657279)
It's just 12B parameters, not 175B. You can probably use int16 or int8.
Re:Get answers for Shell Commands from GPT3 right from your ...
by moffkalast on Friday November 18, 2022 @07:33PM
(#33665171)
That's afaik already the size when using half precision. And it's not like GPT3 doesn't get half queries wrong as-is in its optimal form, detracting from it will just make it unusable. Check out GPT-J if you want to see a functionally useless model that can be run locally.
It's not like this is something that can be surmounted without extra storage, neural nets are just a fancy system for compression and data access. You can either go for less data or more compression but either way your results won't ever be as good.