News for hackers. Stuff that matters.
Re:Llama.cpp: Full CUDA GPU Acceleration by b33j0r on Monday June 12, 2023 @11:29PM (#36304815)
Not quite. Nothing to do directly with python. This was the introduction of 4 and 8 bit quantization to a large number of people.

There wasn’t a python library like that anyone was used to using. Would have always been a C extension anyway.

Starting with the cpu in this situation made sense, strangely. There are python wrappers now. I tried to make one for ya’ll in rust in April, but haha I had a compiler issue I never solved.

There are no comments on this comment yet.