News for hackers. Stuff that matters.

A comment on GigaToken: ~1000x faster Language model tokenization. Parent comment

Re:GigaToken: ~1000x faster Language model tokenization by fwip on Wednesday July 22, 2026 @02:15PM (#49011068)
That's fair, I just figure there are useful scenarios as well. Apologies if I came off as dismissive!
[ Parent | on HN ]
Re:GigaToken: ~1000x faster Language model tokenization by ac2u on Wednesday July 22, 2026 @02:33PM (#49011336)
It didn’t come off as dismissive to me. I was curious as well as to where such optimizing helps and knew that the answers to your question would help me discover use cases I didn’t think of
[ Parent | on HN ]