Re:OpenAI's plans according to sama
by sheepscreek on Wednesday May 31, 2023 @07:04PM
(#36145649)
This is what I understood as well. They want to either democratize adoption or not release it. The last thing they/anyone wants is for another BigCo or Govt to h take undue advantage of the model (through fine-tuning?) when others cannot.
That said, I can imagine a GPTQ/4-bit quantized model to be smaller and easier to run on somewhat commodity clusters?
Or it could run with GGML/llama.cpp on a cloud instance with a TB of RAM?
After seeing what people were able to do with LLaMA, I am positive that the community will find a way to run it - albeit with some loss in performance.
It would be truly amazing if they used their computing to develop quantized models as well.
That said, I can imagine a GPTQ/4-bit quantized model to be smaller and easier to run on somewhat commodity clusters?
Or it could run with GGML/llama.cpp on a cloud instance with a TB of RAM?
After seeing what people were able to do with LLaMA, I am positive that the community will find a way to run it - albeit with some loss in performance.
It would be truly amazing if they used their computing to develop quantized models as well.