News for hackers. Stuff that matters.

A comment on Releasing Persimmon-8B. Parent comment

Re:Releasing Persimmon-8B by swyx on Thursday September 07, 2023 @07:37PM (#37427189)
its not so much about benefit, as it is a design goal to want large context windows.

https://twitter.com/suchenzang/status/1699926157028897078?s=... notes some issues directly comparing the 16k context number. the odd choice of tokenizer means its effectively like a 10-12k model (? ballpark, not calculated)

[ Parent | on HN ]
Re:Releasing Persimmon-8B by craigacp on Thursday September 07, 2023 @08:30PM (#37427683)
There's a correction to that tweet, larger vocab means fewer tokens for any given sequence (usually, assuming it's not to add other languages or character sets).
[ Parent | on HN ]
Re:Releasing Persimmon-8B by euclaise on Friday September 08, 2023 @06:38PM (#37440196)
That tweet had it backwards, more tokens in tokenizer means that the 16k token context window typically allows for even longer passages than if LLaMA were 16k
[ Parent | on HN ]