Re:OpenAI's plans according to sama
by oceanplexian on Wednesday May 31, 2023 @09:18PM
(#36146581)
I'm running Wizard-Vicuna 30B on a Nvidia 3090, with an optimized version of GPTQ at 10 tokens/sec and it's at least 80-85% as capable as GPT3.5. 95% as capable for tasks such as: classification, story writing, scraping, writing comments, turning unstructured data into structured results.
In fact I wrote a little app to control my smart home, literally dropped in a fake OpenAI endpoint for the same app and it worked 100% as well as OpenAI did with the same prompts. If you are on the bleeding edge it's obvious this stuff is moving like a freight train. YES, we are going to be able to run it ourselves, on local hardware.
In fact I wrote a little app to control my smart home, literally dropped in a fake OpenAI endpoint for the same app and it worked 100% as well as OpenAI did with the same prompts. If you are on the bleeding edge it's obvious this stuff is moving like a freight train. YES, we are going to be able to run it ourselves, on local hardware.