This is what I understood as well. They want to either democratize adoption or not release it. The last thing they/anyone wants is for another BigCo or Govt to h take undue advantage of the model (through fine-tuning?) when others cannot.

That said, I can imagine a GPTQ/4-bit quantized model to be smaller and easier to run on somewhat commodity clusters?

Or it could run with GGML/llama.cpp on a cloud instance with a TB of RAM?

After seeing what people were able to do with LLaMA, I am positive that the community will find a way to run it - albeit with some loss in performance.

It would be truly amazing if they used their computing to develop quantized models as well.

I'm running Wizard-Vicuna 30B on a Nvidia 3090, with an optimized version of GPTQ at 10 tokens/sec and it's at least 80-85% as capable as GPT3.5. 95% as capable for tasks such as: classification, story writing, scraping, writing comments, turning unstructured data into structured results.

In fact I wrote a little app to control my smart home, literally dropped in a fake OpenAI endpoint for the same app and it worked 100% as well as OpenAI did with the same prompts. If you are on the bleeding edge it's obvious this stuff is moving like a freight train. YES, we are going to be able to run it ourselves, on local hardware.

If you really need, 170B parameter model can infer a few tokens per minute on commodity hardware.

A big chunk of developments based on Facebook’s LLaMA model are by small-time developers and individuals, not large players. Facebook has already shown a viable way to release models in the way you described.