News for hackers. Stuff that matters.
Re:DeepDive in everything of Llama3: revealing detailed insi... by moffkalast on Saturday February 22, 2025 @07:28AM (#43138401)
It's more than just that, practically every notable open model released in the past year or so uses tiktoken as the tokenizer.

There are no comments on this comment yet.