News for hackers. Stuff that matters.
Re:21.2× faster than llama.cpp? plus 40% memory usage reduction by x0n on Wednesday June 12, 2024 @06:27PM (#40663880)
The hyperbollocks marketingspeak in the summary paragraph put me off:

"The key insight of PowerInfer-2 is to utilize the heterogeneous computation, memory, and I/O resources in smartphones by decomposing traditional matrix computations into fine-grained neuron cluster computations. Specifically, PowerInfer-2 features a polymorphic neuron engine that adapts computational strategies for various stages of LLM inference. Additionally, it introduces segmented neuron caching and fine-grained neuron-cluster-level pipelining, which effectively minimize and conceal the overhead caused by I/O operations."

Ahem, what? Let's overload a biological construct "neuron" to imbue it with magical technopowers and then derive the rest of our BS from this. No sale.

Re:21.2× faster than llama.cpp? plus 40% memory usage reduction by wakawaka28 on Wednesday June 12, 2024 @07:20PM (#40664257)
"Neuron" is not out of place when you're talking about software that uses neural networks...
Re:21.2× faster than llama.cpp? plus 40% memory usage reduction by samatman on Wednesday June 12, 2024 @06:34PM (#40663935)
Processes can't have children either, since, being a bunch of arithmetic, they lack genitalia. You can't kill them, because they aren't alive. They can't even sleep(1), another biological process fast arithmetic has no use for.

When will the false advertising end?!?