News for hackers. Stuff that matters.

A comment on Show HN: How LLMs Work – Interactive visual guide based on Karpathy's lecture. Parent comment

Re:Show HN: How LLMs Work – Interactive visual guide based o... by ynarwal__ on Friday April 24, 2026 @10:54AM (#47891139)
I appreciate the feedback, I did notice that as well and I had this thought perhaps this is not worth fixing since I have a link to tiktokenizer. I decided to remove it and just added a more prominent link to tiktokenizer.
[ Parent | on HN ]
Re:Show HN: How LLMs Work – Interactive visual guide based o... by thesz on Saturday April 25, 2026 @06:06AM (#47900177)
BPE that is used in tokenization is very simple: https://en.wikipedia.org/wiki/Byte-pair_encoding
[ Parent | on HN ]