Re:GPT-3 tries pickup lines
by gattilorenz on Wednesday March 24, 2021 @02:42AM
(#26564215)
> The result is surprisingly good
"The result" as in "the first one"? The rest is almost Markov chain level...
It's "surprising" since in the article, the 2.7B parameter model (Ada) didn't even produce sentences, while the result from 2.7B GPT Neo is closer in quality to what was output from GPT-3 Babbage (6.7B parameters).
As you can see from no. 10, GPT Neo did remember the context of trying to come up with pick-up lines. The fact that some of the lines feel "Markov chain level" is likely caused by the network's bad understanding of what pick-up lines are. Pick-up lines are a rather difficult concept, which is what the OP article tries to demonstrate.
Why do you say Ada is 2.7b ?
The sizes of the four GPT-3 variants were shared on Reddit by Stella Athena, one of the researchers behind GPT Neo: https://www.reddit.com/r/MachineLearning/comments/ma9kaw/p_e...
1. How did this little blossom happen? When are you going to bloom? 2. Have you ever thought about a dark horse in the running for Miss November? 3. You can spot the man who loves me by my neck – and he definitely knows it. 4. Are there any lucky cats who get to sleep in my bed every night? 5. Are black and whites everywhere? Running for the hills 6. My younger brother and I used to play a game. He’d pretend to be a bull in the pasture, and I’d pretend to be the one being held. 7. Why was this movie rated PG? Because it’s rated PG. 8. When is the last time you had to see a movie in children’s theaters? 9. You are so sexy I would hate for anyone to see you down here 10. I’d love to sleep with you right now, but I have a child with me