Re:Effort – a possibly new algorithm for LLM Inference
by dhruvdh on Wednesday April 17, 2024 @02:03PM
(#40068134)
I would imagine the importance of weights depends on the prompt. How do you decide which weights are important?
[ Parent
| on HN ]
Re:Effort – a possibly new algorithm for LLM Inference
by kolinko on Wednesday April 17, 2024 @02:10PM
(#40068214)
Yeah, that is the point more or less - it dynamically chise the weights layer per layer depending on the internal state.
A bit technical explaination here. https://kolinko.github.io/effort/equations.html