News for hackers. Stuff that matters.
Re:Show HN: Speeding up LLM inference 2x times (possibly) by kolinko on Wednesday April 17, 2024 @06:09PM (#40070590)
The algorithm still uses all the weights, just not all the time - just skips the weights when they are not important given an input vector.

Also, approximation methods, as a field, are not new and they have shown their use.

Having said all that, extraordinary claims require extraordinary evidence - that’s why I hedge the communication messages. It’s „probably” until we get serious tests going on

There are no comments on this comment yet.