Re:21.2× faster than llama.cpp? plus 40% memory usage reduction
by wakawaka28 on Wednesday June 12, 2024 @07:20PM
(#40664257)
"Neuron" is not out of place when you're talking about software that uses neural networks...
Re:21.2× faster than llama.cpp? plus 40% memory usage reduction
by samatman on Wednesday June 12, 2024 @06:34PM
(#40663935)
Processes can't have children either, since, being a bunch of arithmetic, they lack genitalia. You can't kill them, because they aren't alive. They can't even sleep(1), another biological process fast arithmetic has no use for.
When will the false advertising end?!?
"The key insight of PowerInfer-2 is to utilize the heterogeneous computation, memory, and I/O resources in smartphones by decomposing traditional matrix computations into fine-grained neuron cluster computations. Specifically, PowerInfer-2 features a polymorphic neuron engine that adapts computational strategies for various stages of LLM inference. Additionally, it introduces segmented neuron caching and fine-grained neuron-cluster-level pipelining, which effectively minimize and conceal the overhead caused by I/O operations."
Ahem, what? Let's overload a biological construct "neuron" to imbue it with magical technopowers and then derive the rest of our BS from this. No sale.