News for hackers. Stuff that matters.

A comment on Llama-CPU: Fork of Facebooks LLaMa model to run on CPU.

Re:Llama-CPU: Fork of Facebooks LLaMa model to run on CPU by visarga on Wednesday March 08, 2023 @02:42AM (#35066332)
It's useless before the model gets instruction and preference tunings. Won't even follow a simple ask, it will just assume it is a list of questions and generate more, or continue with slightly related comments.

FB trained a LLaMA-I (instruction tuned) variant for sports, just to show they can, but I don't think it got released.

[ Parent | on HN ]
Re:Llama-CPU: Fork of Facebooks LLaMa model to run on CPU by ma2rten on Wednesday March 08, 2023 @03:41AM (#35066702)
It's still useful, but you need to know how to use it.
[ Parent | on HN ]
Re:Llama-CPU: Fork of Facebooks LLaMa model to run on CPU by ShamelessC on Wednesday March 08, 2023 @03:40AM (#35066696)
Useless!? C'mon.
[ Parent | on HN ]
Re:Llama-CPU: Fork of Facebooks LLaMa model to run on CPU by notpushkin on Wednesday March 08, 2023 @03:47AM (#35066737)
So, you need to know how to tune it.
[ Parent | on HN ]
Re:Llama-CPU: Fork of Facebooks LLaMa model to run on CPU by qingdao99 on Wednesday March 08, 2023 @04:07AM (#35066846)
Surely it would work with a format like:

User: <question or task> Assistant:

[ Parent | on HN ]
Re:Llama-CPU: Fork of Facebooks LLaMa model to run on CPU by cfcf14 on Wednesday March 08, 2023 @05:12AM (#35067229)
You have to prompt it correctly, non-instruction-aligned models don't behave like agent simulators by default.
[ Parent | on HN ]
Re:Llama-CPU: Fork of Facebooks LLaMa model to run on CPU by RugnirViking on Wednesday March 08, 2023 @10:13AM (#35069949)
it's not that useless, you just have to prompt it the right way (usually by offering an example of the kind of output you want)
[ Parent | on HN ]
Re:Llama-CPU: Fork of Facebooks LLaMa model to run on CPU by skhm on Wednesday March 08, 2023 @03:00PM (#35073980)
Try giving it a simple request instead ;)
[ Parent | on HN ]