Re:GigaToken: ~1000x faster Language model tokenization
by marcelroed on Wednesday July 22, 2026 @02:59PM
(#49011741)
I can add some benchmarks for compatibility mode in the future. I have a little more juice to squeeze out of the Python interop though, so not quite ready for it yet.
[ Parent
| on HN ]
There are no comments on this comment yet.