Re:Mass Editing Memory in a Transformer
by drdeca on Friday April 21, 2023 @06:46PM
(#35661268)
Would the natural analogy of “tokens” be “words”, or something more like, “portion of mouth-movement”?
[ Parent
| on HN ]
I more imagine someone speaking another language, when I break down written phonemes for tokenization purposes.
As a native English speaker who grew up around code-switching Spanish speakers, I often heard speech that sounded really fast.
But I was hearing “hay-un-a-al-pa-ca-per-si-gui-en-do-un tren-de-car-ga”
I heard that as individual tokens, while my friends simply heard “¡There is an alpaca chasing that freight train!”