200k emails is not enough to train a model from scratch. If you check out the google colab file in the GPT-Neo repository, it talks about how to fine-tune the model on data which is what you want to do

You might get some really promising results with finetuning.

If anything, you could build writing assistance that almost automates responses.

I've been co-authoring a library that lets you finetune such models in a single line of code.

https://github.com/backprop-ai/backprop

In specific the text generation finetuning example should be what you are looking for: https://github.com/backprop-ai/backprop/blob/main/examples/F...

Hope this helps, happy to chat more about it. Pretty curious about the results.

I wouldn't trust any model to generate text for customers yet. Not even the largest GPT3. There are no guarantees on what they will output and could be damaging to your business.

You're better off either: 1- Defining common "intents" that a lot of customer queries are categorized into, and having a model map the incoming message to the appropriate canned response. Look at Rasa, for an example of this.

2- if you insist on generating the text, have it be a recommendation to a human agent that either chooses to send it or writes their own response.

Thanks for the advice.

You fine-tune an existing pretrained model on your proprietary dataset.