Re:Show HN: TokenDagger – A tokenizer faster than OpenAI's T...
by serjester on Monday June 30, 2025 @01:44PM
(#44426006)
Tokenizing text is ridiculously small part of the overall computation that goes into serving a request. With that said if you’re doing this on petabytes of data, never hurts to have something faster.
[ Parent
| on HN ]