Re:Announcement: AI generated answers are officially banned ...
by nl on Wednesday January 04, 2023 @05:34PM
(#34252886)
> Technically, they aren't trained that way, their opposite recognition AI is trained that way.
No, they are trained that way. These aren't GANs or anything resembling that. There is a reinforcement model that is build from expert human guidance which is then used to fine-tune the LM.
From the HuggingFace explainer page I linked above: [there are] three core steps:
1. Pretraining a language model (LM),
2. gathering data and training a reward model, and
3. fine-tuning the LM with reinforcement learning.
Read https://arxiv.org/abs/2203.02155 if you want all the details (although the HF explainer is easier to follow). From the abstract of that paper:
> Starting with a set of labeler-written prompts and prompts submitted through the OpenAI API, we collect a dataset of labeler demonstrations of the desired model behavior, which we use to fine-tune GPT-3 using supervised learning. We then collect a dataset of rankings of model outputs, which we use to further fine-tune this supervised model using reinforcement learning from human feedback.
No, they are trained that way. These aren't GANs or anything resembling that. There is a reinforcement model that is build from expert human guidance which is then used to fine-tune the LM.
From the HuggingFace explainer page I linked above: [there are] three core steps:
1. Pretraining a language model (LM),
2. gathering data and training a reward model, and
3. fine-tuning the LM with reinforcement learning.
Read https://arxiv.org/abs/2203.02155 if you want all the details (although the HF explainer is easier to follow). From the abstract of that paper:
> Starting with a set of labeler-written prompts and prompts submitted through the OpenAI API, we collect a dataset of labeler demonstrations of the desired model behavior, which we use to fine-tune GPT-3 using supervised learning. We then collect a dataset of rankings of model outputs, which we use to further fine-tune this supervised model using reinforcement learning from human feedback.
There is no "opposite recognition AI".