Re:Stop Calling Everything AI, Machine-Learning Pioneer Says
by randomdata on Thursday January 19, 2023 @12:31AM
(#34437036)
There were some papers linked here yesterday. I'm not good at memorizing URLs. Sorry.
Do you have a source for that? I've implemented GPT from scratch and I don't see anywhere that doesn't just take the current output + input as attention and produces the next word. The loss being if it correctly guessed what would be the next word according to context and position.