Re:TL;DR of Deep Dive into LLMs Like ChatGPT by Andrej Karpathy
by wolfhumble on Monday February 10, 2025 @07:14AM
(#42999401)
I haven't watched the video, but was wondering about the Tokenization part from the TL;DR:
"|" "View" "ing" "Single"
Just looking at the text being tokenized in the linked article, it looked like (to me) that the text was: "I View", but the "I" is actually a pipe "|".
"|" "View" "ing" "Single"
Just looking at the text being tokenized in the linked article, it looked like (to me) that the text was: "I View", but the "I" is actually a pipe "|".
From Step 3 in the link that @miletus posted in the Hacker News comment: https://x.com/0xmetaschool/status/1888873667624661455 the text that is being tokenized is:
|Viewing Single (Post From) . . .
The capitals used (View, Single) also makes more sense when seeing this part of the sentence.