Re:GPT-Neo – Building a GPT-3-sized model, open source and free
by ravi-delia on Monday January 18, 2021 @09:34AM
(#25822141)
Op was clearly not talking about Alpha Zero, a different technology made by different people for a different purpose. Instead, they were noting that despite displaying some truly excellent world modeling, GPT-3 is trained on data that encourages it to vomit up rehashes. It's very possible that the next generation will overcome this and wind up completely holding together long-run concepts and recursion, at least if scaling parameters keeps working, but for now it is a real limitation.
GPT-3 writes like a sleepy college student with 30 minutes before the due date; with shockingly complete grasp of language, but perhaps not complete understanding of content. That's not just an analogy, I am a sleepy college student. When I write an essay without thinking too hard it displays exactly the errors that GPT-3 makes.
Re:GPT-Neo – Building a GPT-3-sized model, open source and free
by tbenst on Monday January 18, 2021 @09:34AM
(#25822143)
GPT-3 can’t play Go.
Re:GPT-Neo – Building a GPT-3-sized model, open source and free
by Tenoke on Monday January 18, 2021 @11:59AM
(#25823745)
It almost definitely can to some extent given that gpt2 could play chess [0].
0. https://slatestarcodex.com/2020/01/06/a-very-unlikely-chess-...
1) Please play a winning game of Go against Alpha Zero, just by googling the topic.
2) Next please explain how Alpha Zero’s game’s could forever change Go opening theory[2], without any genuine creativity.
[1] that “the output from GPT-3, DALL-E, et al is similar to what you get from googling the prompt and stitching together snippets from the top results.”
[2]”Rethinking Opening Strategy: AlphaGo's Impact on Pro Play” by Yuan Zhou