Re:Generating Fake Conversations by Fine-Tuning OpenAI's GPT...
by gwern on Thursday March 14, 2019 @10:16AM
(#19389388)
As it happens, nshepperd ran his finetuning GPT-2-small on our IRC channel. I'd tried it before with char-RNN back in 2015 or so, and I have to say, GPT-2-small trained way faster and better than my IRC char-RNN did.
The samples also looked a lot better than OP's. I assume that's because he ran it for more like a day on a few hundred MB of chat logs.
Re:Generating Fake Conversations by Fine-Tuning OpenAI's GPT...
by gwern on Friday March 15, 2019 @09:20AM
(#19399395)
I didn't run on IRC because my GPUs were busy with poetry, FWIW: https://www.gwern.net/RNN-metadata#finetuning-the-gpt-2-smal... https://slatestarcodex.com/2019/03/14/gwerns-ai-generated-po... http://sevensecularsermons.org/on-the-significance-of-gwerns...
Re:Generating Fake Conversations by Fine-Tuning OpenAI's GPT...
by holler on Thursday March 14, 2019 @11:08AM
(#19389881)
found this really interesting, I wonder if it could be used to create ~realistic conversations on a new discussion platform so it appears like people are there? :)
Re:Generating Fake Conversations by Fine-Tuning OpenAI's GPT...
by oneshot908 on Thursday March 14, 2019 @03:13PM
(#19392525)
Who's going to connect this to CleverBot and let the hilarity ensue?
Also, I wouldn't say the results are nonsensical - I think it has learned a lot more than a markov chain or a simple rnn but I agree that especially on the surface they dont even sound like they surpass Eliza by much. Moreover, it is significantly more apparent how much it learns about the different people you've talked to AFTER you run it on your own data.
For a somewhat more novel/interesting result with fine-tuning GPT, I can recommend checking out gwern's post[1] on training it on a big poetry corpus.
1. https://www.gwern.net/RNN-metadata#finetuning-the-gpt-2-smal...