News for hackers. Stuff that matters.
Re:Effort – a possibly new algorithm for LLM Inference by dhruvdh on Wednesday April 17, 2024 @02:03PM (#40068134)
I would imagine the importance of weights depends on the prompt. How do you decide which weights are important?
Re:Effort – a possibly new algorithm for LLM Inference by kolinko on Wednesday April 17, 2024 @02:10PM (#40068214)
Yeah, that is the point more or less - it dynamically chise the weights layer per layer depending on the internal state.

A bit technical explaination here. https://kolinko.github.io/effort/equations.html