Re:Teaching GPT-3 to reverse words
by rprenger on Monday May 16, 2022 @05:10PM
(#31403175)
I don't think a larger vocab would help. All the individual letters are in the ~50k token vocab already, but the word "alphabet" will still not get tokenized to [a, l, p, h, a, b, e, t]. Using a larger vocab like PaLM's 256k vocab would have the same issue.
[ Parent
| on HN ]
There are no comments on this comment yet.