Re:OpenGPT-2: We Replicated GPT-2 Because You Can Too
by dgacmu on Friday August 23, 2019 @10:18AM
(#20777875)
I use it for http://nametango.ai/ (a startup/product name generation assistant). There are more details on my page, but in a nutshell, it takes about 7 seconds to generate a result on CPU (I generate a batch of 40 results for every query and de-dup them), and it runs in about 300 milliseconds on a Titan V GPU.
That is, however, a very batch friendly application, so a serial generation may perform somewhat better on CPU only.
(I'm comparing to dual xeon gold 16 core CPUs). When using the GPU to generate the GPT-2 results, the CPU is mostly bored.
(responding to a comment that someone deleted, yes, it does suggest a lot of existing names, I haven't yet added something to filter out stuff that already exists... noted, and suggestions appreciated, but this probably isn't the thread for it! Feel free to drop me a note)
That is, however, a very batch friendly application, so a serial generation may perform somewhat better on CPU only.
(I'm comparing to dual xeon gold 16 core CPUs). When using the GPU to generate the GPT-2 results, the CPU is mostly bored.
(responding to a comment that someone deleted, yes, it does suggest a lot of existing names, I haven't yet added something to filter out stuff that already exists... noted, and suggestions appreciated, but this probably isn't the thread for it! Feel free to drop me a note)