News for hackers. Stuff that matters.
Re:2D Tokenization for Large Language Models by bubble_boi on Tuesday May 02, 2023 @05:58AM (#35785313)
This article is about how we process text before passing it to large language models, the problems with this approach, and an alternative solution.