News for hackers. Stuff that matters.

A comment on LLaMA: A foundational, 65B-parameter large language model. Parent comment

Re:LLaMA: A foundational, 65B-parameter large language model by bitRAKE on Friday February 24, 2023 @03:47PM (#34929907)
The 1.4T tokens are what the model was trained on, and not the token range of the embedding.
[ Parent | on HN ]
Re:LLaMA: A foundational, 65B-parameter large language model by dentalperson on Friday February 24, 2023 @04:17PM (#34930264)
Ah, that makes more sense, thank you. Since this was mentioned in the tokenizer section and the number of unique tokens wasn't mentioned I misunderstood.
[ Parent | on HN ]