News for hackers. Stuff that matters.
Re:Intel CEO: 'The entire industry is motivated to eliminate... by ebb_earl_co on Thursday December 14, 2023 @05:10PM (#38648019)
Can you expand on this? This is my first time seeing this guy’s work
Re:Intel CEO: 'The entire industry is motivated to eliminate... by pbronez on Thursday December 14, 2023 @05:30PM (#38648261)
He’s the main developer of Llama.cpp, which allows you to run a wide range of open-weights models on a wide range of non-NVIDIA processors.
Re:Intel CEO: 'The entire industry is motivated to eliminate... by fl0id on Thursday December 14, 2023 @08:40PM (#38650068)
but it's all inference, and most of Nvidias moat is in training afaik.
Re:Intel CEO: 'The entire industry is motivated to eliminate... by refulgentis on Thursday December 14, 2023 @11:07PM (#38650879)
People have really bizarre overdramatic misunderstandings of llama.cpp because they used it a few times to cook their laptop. This one really got me giggling though.
Re:Intel CEO: 'The entire industry is motivated to eliminate... by ilaksh on Friday December 15, 2023 @01:44AM (#38651657)
I am integrating llama.cpp into my application. I just went through one of their text generation examples line-by-line and converted it into my own class.

This is a leading-edge software library that provides a huge boost for non-Nvidia hardware in terms of inference capability with quantized models.

If you don't understand that, then you have missed an important development in the space of machine learning.

Re:Intel CEO: 'The entire industry is motivated to eliminate... by refulgentis on Friday December 15, 2023 @01:51PM (#38657244)
Jeez. Lol.

At length:

- yes, local inference is good. I can't say this strongly enough: llama.cpp is a fraction of a fraction of local inference.

- avoid talking down to people and histrionics. It's a hot field, you're in it, but like all of us always, you're still learning. When faced with a contradiction, check your premises, then share them.

Re:Intel CEO: 'The entire industry is motivated to eliminate... by ilaksh on Friday December 15, 2023 @01:39AM (#38651636)
There is an example of training https://github.com/ggerganov/llama.cpp/tree/1f0bccb27929e261...

But that's absolutely false about the Nvidia moat being only training. Llama.cpp makes it far more practical run inference on a variety of devices. Including ones with or without Nvidia hardware.