Re:PruningRadixTrie – 1000x faster Radix trie for prefix sea...
by meepmorp on Saturday September 30, 2023 @12:17PM
(#37716887)
I've seen 1.5 chars/token used as a rule of thumb for estimating token counts in Chinese text.
[ Parent
| on HN ]
There are no comments on this comment yet.