News for hackers. Stuff that matters.

A comment on Mistral 7B, the most powerful language model for its size to date, Apache 2.0. Parent comment

Re:Mistral 7B, the most powerful language model for its size... by benkaiser on Wednesday September 27, 2023 @09:22PM (#37683982)
~13 tokens per second. I asked for a story about goldilocks and this was the timings on my M1 air using `ollama run mistral --verbose`:

total duration: 33.964492834s

load duration: 1.471584ms

prompt eval count: 8 token(s)

prompt eval duration: 385.334ms

prompt eval rate: 20.76 tokens/s

eval count: 418 token(s)

eval duration: 31.887033s

eval rate: 13.11 tokens/s

[ Parent | on HN ]

There are no comments on this comment yet.