AWS/Azure are a platform, and they work reliably. What kind of platform 'sometimes works'?
Fine-tune on educational materials, and a language model could be a 24/7 assistant to students.
Fine-tune on psychology or medicine data, and a language model could tell you of all medicine interactions, or act as a better ELIZA, and "socialize with" - and "support" people with depression or trauma.
Fine-tune on etiquette and social norms, and autistic people could ask questions without being ashamed.
These are some positive use cases. There are also neutral use cases (seeding a new social platform with autogenerated comments), and negative use cases (SEO spam, fake reviews, scaling up disinformation campaigns).
Edit: not complaining, but no idea why this was downvoted. Would be helpful to state why, so I don't make this mistake in the future (I am optimizing for useful replies).
Has anyone done it and demonstrated that it can be done on GPT to get useful and reliable output for some domain? If we exclude fake news/ twitter-bots or something equally silly.
For GPT-2 it is as easy as creating a corpus you want to fine-tune on, adding new unseen tokens to the dictionary, and run a finetuning command. https://huggingface.co/transformers/v1.2.0/examples.html#fin...
> Has anyone done it
Many have. AI Dungeon is a great example. Others are experimenting with humor generation, short story synopsis generation/creative writing, and supportive chat bots.
As best I can tell the only thing would be some kind of GMail-like auto-reply which a human consciously edits, and which has little consequence if it's wrong.
Are the models useful for customer service? Like reading a manual or knowledgebase and then answering a customer's questions about a product, and troubleshooting problems? Like about your Android phone, which lots of people have trouble using?
That would be a trillion dollar business. As best as I can tell that's completely beyond GPT-3 and requires a huge breakthrough, which may or may not happen.