News for hackers. Stuff that matters.

A comment on The Bitter Lesson is coming for Tokenization. Parent comment

Re:The Bitter Lesson is coming for Tokenization by rar00 on Wednesday June 25, 2025 @08:33AM (#44376558)
This argument works better for state space models. A transformer would still steps context one token at a time, not maintain an internal 1e18 state.
[ Parent | on HN ]
Re:The Bitter Lesson is coming for Tokenization by mgraczyk on Wednesday June 25, 2025 @12:52PM (#44379401)
That doesn't matter, are you familiar with any theoretical results in which the computation is somehow limited in ways that practically matter when the context length is very long? I am not
[ Parent | on HN ]