News for hackers. Stuff that matters.

A comment on 70B Llama 2 at 35tokens/second on 4090. Parent comment

Re:70B Llama 2 at 35tokens/second on 4090 by behnamoh on Wednesday September 13, 2023 @11:12AM (#37497842)
Wow, the top 14 are all proprietary models (except 2 models which are not commonly used in OS community anyway).

And surprisingly, Llama 33B performs _better_ than Llama 65B!

[ Parent | on HN ]
Re:70B Llama 2 at 35tokens/second on 4090 by NhanH on Wednesday September 13, 2023 @11:18AM (#37497920)
That table is comparing few shot prompting GPT 4 to zero shot llama.
[ Parent | on HN ]