Re:Show HN: TokenDagger – A tokenizer faster than OpenAI's T...
by stavros on Monday June 30, 2025 @11:47AM
(#44424691)
Yeah, if it's not right, it doesn't work.
[ Parent
| on HN ]
Re:Show HN: TokenDagger – A tokenizer faster than OpenAI's T...
by DSingularity on Monday June 30, 2025 @12:06PM
(#44424906)
Not true. Things can work with hacks. Your standards might consider it unacceptable to have hacks. So you can have a “make it right” stage.
Re:Show HN: TokenDagger – A tokenizer faster than OpenAI's T...
by darknoon on Monday June 30, 2025 @11:54AM
(#44424765)
In ML, often it does work to a degree even if it's not 100% correct. So getting it working at all is all about hacking b/c most ideas are bad and don't work. Then you'll find wins by incrementally correcting issues with the math / data / floating point precision / etc.
Re:Show HN: TokenDagger – A tokenizer faster than OpenAI's T...
by gabrielhidasy on Monday June 30, 2025 @01:24PM
(#44425794)
Depends on your definition of "right" and "work". It could be a big ball of mud that always returns exactly the required response (so it 'works'), but be hellish hard change and very picky about dependencies and environment (so it's not 'right').