Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on GitHub Copilot is not infringing copyright. Parent comment
This is a massive simplification. It's adequate for most purposes, but when discussing at this level this simplification breaks down.
Tokens are stored, and the NN contains weights of likelihood of a token occurring after another, given a sequence of prior (and possibly post) tokens.
Verbatim retrieval usually means there is very little variation in the training data for that sequence, so the same set of weights gets stored.
So "retrieval" is actually the same generative process as unique code uses, but the NN hasn't seen any other versions.
There are no comments on this comment yet.
This is a massive simplification. It's adequate for most purposes, but when discussing at this level this simplification breaks down.
Tokens are stored, and the NN contains weights of likelihood of a token occurring after another, given a sequence of prior (and possibly post) tokens.
Verbatim retrieval usually means there is very little variation in the training data for that sequence, so the same set of weights gets stored.
So "retrieval" is actually the same generative process as unique code uses, but the NN hasn't seen any other versions.