News for hackers. Stuff that matters.
Re:OpenLLaMA: An Open Reproduction of LLaMA by jstarfish on Wednesday May 03, 2023 @12:55PM (#35804708)
> What a token is depends on the context of the model you're using, but generally a token is a portion of a word.

When doing quick estimates, I just assume every syllable is a token. It tends to overestimate, which is fine for my OOM mitigation purposes.

There are no comments on this comment yet.