News for hackers. Stuff that matters.

A comment on GPT-3 has no idea what it’s talking about. Parent comment

Re:GPT-3 has no idea what it’s talking about by ClumsyPilot on Saturday August 22, 2020 @09:54PM (#24248907)
I did not downvote, but I think it's because you kinda missed the crux of my question is - what is fine-tuning, like what do you have to actually do?

Has anyone done it and demonstrated that it can be done on GPT to get useful and reliable output for some domain? If we exclude fake news/ twitter-bots or something equally silly.

[ Parent | on HN ]
Re:GPT-3 has no idea what it’s talking about by voces on Sunday August 23, 2020 @06:46PM (#24255933)
> what is fine-tuning, like what do you have to actually do?

For GPT-2 it is as easy as creating a corpus you want to fine-tune on, adding new unseen tokens to the dictionary, and run a finetuning command. https://huggingface.co/transformers/v1.2.0/examples.html#fin...

> Has anyone done it

Many have. AI Dungeon is a great example. Others are experimenting with humor generation, short story synopsis generation/creative writing, and supportive chat bots.

[ Parent | on HN ]