Extremely complex classification task is all I can say

GPT-3 is a generative model, isn't it? Can you explain how you converted GPT-3 to a classification model?

There are a couple of ways to do it. You can give it a prompt that shows examples of the classification and it mimics what it thinks is the correct behavior when you feed it new unclassified input. They also have a search endpoint that lets you do classification by giving it an input along with labels as the searchable documents and using the resulting semantic relevance scores.

You can add new "heads" to GPT networks and train those heads to use GPT for new applications.

Not with GPT-3. I believe only Microsoft is allowed to do that.

In general GPT3 is not SotA on (any?) classification task, did you just not have enough data to fine tune a discriminative transformer model? Inference should be cheaper with a smaller transformer/also less lock-in.

I can't go into too much detail here about why we couldn't do that, but one aspect that we found VERY useful is that GPT-3 could draw on real world knowledge not present in the dataset to enhance the results.