News for hackers. Stuff that matters.

A comment on Alpaca- An Instruct Tuned Llama 7B. Responses on par with txt-DaVinci-3. Demo up. Parent comment

Re:Alpaca- An Instruct Tuned Llama 7B. Responses on par with... by Taek on Monday March 13, 2023 @10:22PM (#35146534)
Definitively, 13B with quantization will perform better. 4bits has been shown to be the optimal quantization for accuracy vs memory requirements.
[ Parent | on HN ]
Re:Alpaca- An Instruct Tuned Llama 7B. Responses on par with... by thot_experiment on Tuesday March 14, 2023 @04:32AM (#35148719)
Yeah, 7b vs 13b is basically no comparison in any situation, 16bit 7b is def worse than 4bit 13b. I'll be looking into 30B tomorrow. I may be able to do a full matrix of tests 4-16bit X 7-30b.
[ Parent | on HN ]