Only if you're OpenAI

Fine tuning of GPT3 models is available via their public API. Costs credits, and you need to get their permission to use it in an actual application, but it’s not locked in a lab.

So “Only if you’re OpenAI” :)

If the weights were public, the community would figure out a way to fine tune it.

It’s not a matter of ‘figuring out’. The model supports fine tuning. It’s a core feature of the openai API. Running ‘fine tuned’ versions of GPT-3 that are created by customers is literally their SaaS model. They have examples in the documentation. Here: https://help.openai.com/en/articles/5528730-fine-tuning-a-cl...

I get that. But you don’t get the weights. Which means they limit what you can and can’t generate.

A friend of mine was going to build a writing assistant on top of GPT-3. She got a lot of encouragement from them. Then one day the social media storm hit OpenAI, and suddenly safety became a non-negotiable feature of their api. And along with that came the restriction “absolutely no products that can generate unlimited amounts of text, even if you can pay for the credits.”

Poof, no more business.

Imagine buying a keyboard that restricted what you could type.

All of these problems go away when you have access to the weights.

GPT-3 can be quite adaptive given prompt engineering and the uploading of sample files.

Have you used GPT-3 with any of the methods mentioned in the docs?

I’ve seen that GPT-3 can produce quite starkly different results when prompted differently and when samples have been uploaded.