The way its typically done, AFAIK, is that you train these big models on a breadth of information, hoping that it picks up on the generalities of the information. In the case of LLMs, things like basic inference, for example. You then take these big, general models and “fine tune” them for specific applications, with specific bits of data. This way, you get things like basic inference, and logic, while still having something that can answer specific questions.

OpenAI offers the ability to fine-tune some of their models: https://platform.openai.com/docs/guides/fine-tuning

There are also other services that will fine-tune an LLM, for a specific domain, for you: https://activechat.ai/build-your-own-chatgpt/

Smaller models are typically dumber. Sure you could fine-tune a smaller model on say the medical domain and they might even perform better on some benchmarks but they won't reason or generalize as well. domain finetuned large models >>> domain finetuned small models. And because competence in one area bleeds over to other areas, you often need much less domain specific data to finetune on compared to the smaller models.

You can see instances of this with Minerva, where the finetuned 540b version beats the finetuned 62b version despite being finetuned on only a quarter of the data the 62b version was finetuned on.

They're claiming the 13b model beats GPT-3 175b which is an extraordinary claim requiring extraordinary evidence. If that's true, though, it'd be interesting to see if that also applies to fine-tuning. Since the claim is predicated on the 13b model being better trained (amongst other things?), I wonder if limited fine-tuning data handicaps the 13b model even if the base model can outperform GPT-3 Davinci, given your point about large models handling fine-tuning better with limited data.

I mean the benchmarks are there. Can't exactly fake that. It should apply to fine-tuning. fine-tuning works off the back of the weights. That's why instruction-finetuned models even of small models like the T5 converge much faster on any additional fine-tuning or training than their non instruct counterparts as per the flan paper. Honestly, what i'm taking from this paper is that even chinchilla is undertrained. 13B was trained on 1T tokens.

My opinion is entirely opposite - we need conversational AI for exploring the multitude of possibilities and identify what works, what doesn't work, what is/isn't useful given the obvious limitations on accuracy, truthfulness and interpretability; but once we can identify a specific narrow use case then we can fine-tune a ML system for that isn't conversational but is able to provide better results with whatever domain-specific structure is required (linking to specific sources, including external structured data, providing certainty metrics, filtering results according to domain-specific criteria instead of the conversational political correctness filters which fail some domains, treatment info which was correct but has become outdated, etc, etc) that can be done better in non-conversational systems.

The P in GPT is for "pretrained".

The large pretrained model is a prereq for domain-specific models.

Not necessarily, you don't need to make a domain-specific model from a general model, you can definitely make a large pretrained domain-specific model from scratch by training it only on domain-specific data, which can result in a smaller and more efficient model.

If you want to build a domain-specific classifier that determines whether an image is a dog or a cat, and you have 50 labeled images of dogs and cats, it's much better to start with a large model pretrained on millions of images, and then specialize it by training on 50 images of dogs and cats.

Try to start with a NN and 50 images of dogs and cats, and it won't work very well.

Sure, that's correct, but that's absolutely unrelated to what we were talking about; your example is about the general concept of transfer learning to task-specific annotated data, not about domain-specific pretrained models.

For example, if you want a domain-specific model for the legal domain, then you can pre-train a large self-supervised model on every single legal-related document in the world you can get your hands on, instead of a general mix of news and fiction and blogs and everything else - and that might be a more efficient starting point for however many(few) annotated examples you have for your task-specific classifier than the general model.

Legal-related documents are a minuscule fraction of the corpus the large model is trained on. The resulting model won't have the conceptual fluency that the large model has. It's like training a human baby with legal briefs and expecting her to be a good lawyer.

These models are only capable of coherent conversation in the first place because they are so large. As soon as you shrink it down to be 'domain specific' its ability to form coherent sentences even within its own domain greatly reduces.

There are definitely very solid attempts at least to make LLMs that encode biomedical knowledge such as BioGPT which is trained on Pubmed and other domain specific areas. Source: https://arxiv.org/abs/2210.10341

You can already do that. Take your task first to GPT-3 and collect a bunch of outputs. Then fine-tune a small model on them. Works well, but you need to extend the dataset to cover all edge cases because the small model can't draw on the vast knowledge GPT-3 has.

I think you would still need a large model to train on general-purpose knowledge and then train on domain specific things to get the specialized knowledge to be truly useful. For example, without a general-purpose model, if someone wanted domain-specific language "translated" for a neophyte, I doubt it would be able to without having also been trained on a general-purpose dataset