part of it is attention, right. if it sees the word pirate then it's more likely to say Argh and Matey...

and because it's transformers onto of transformers it can says phrases based on words and phrases

the smaller ones like to repeat themselves I'm not sure if they Reinforcement learning is what fixed that or just a larger model.