This is a massive simplification. It's adequate for most purposes, but when discussing at this level this simplification breaks down.

Tokens are stored, and the NN contains weights of likelihood of a token occurring after another, given a sequence of prior (and possibly post) tokens.

Verbatim retrieval usually means there is very little variation in the training data for that sequence, so the same set of weights gets stored.

So "retrieval" is actually the same generative process as unique code uses, but the NN hasn't seen any other versions.