News for hackers. Stuff that matters.
Re:1-Bit LLMs Could Solve AI's Energy Demands by renonce on Friday May 31, 2024 @10:05AM (#40535352)
For a 4096x4096 matrix its symmetry group has size 4096!. The Stirling’s approximation gives ln(4096!)=4096ln(4096)-4096 which is about 10~11 bits per 4096 numbers. This is less than 0.003 bit per parameter saved.

There are no comments on this comment yet.