Re:Show HN: I made a free-to-use GPT-2 API
by rahimnathwani on Tuesday September 22, 2020 @10:05PM
(#24562207)
Related, Fabrice Bellard developed a C program for GPT-2 inference. It's not open source, but the binary is free to download, and can use any pretrained GPT-2 model:
I tried it using a GPT-2 model I'd finetuned, and it worked well.
I started making a simple chatbot with it using Python: at each step, I fed in the conversation so far, and a speaker prompt (e.g. PERSON2:), and read the output until I saw PERSON1: appear. Rinse and repeat.
Unfortunately, I got stuck with some text encoding issue between python and the CLI binary. I tried obvious things like forcing the decode to use UTF-8, but after a couple of steps I always ended up receiving some characters that weren't valid.
I didn't have this problem when running the binary from the command line, so it must have been something to do with how I was using python popen.
https://bellard.org/nncp/gpt2tc.html
I tried it using a GPT-2 model I'd finetuned, and it worked well.
I started making a simple chatbot with it using Python: at each step, I fed in the conversation so far, and a speaker prompt (e.g. PERSON2:), and read the output until I saw PERSON1: appear. Rinse and repeat.
Unfortunately, I got stuck with some text encoding issue between python and the CLI binary. I tried obvious things like forcing the decode to use UTF-8, but after a couple of steps I always ended up receiving some characters that weren't valid.
I didn't have this problem when running the binary from the command line, so it must have been something to do with how I was using python popen.