News for hackers. Stuff that matters.
Re:1-Bit LLMs Could Solve AI's Energy Demands by raverbashing on Friday May 31, 2024 @05:48AM (#40533093)
Quantization might need a 'relaxing' or 'normalizing' step before to improve results, it would be an interesting research area

Quantize each block or vector as a unit, bound to some metric instead of simply quantizing the whole thing all at once.

It's an interesting topic https://old.reddit.com/r/LocalLLaMA/comments/1ba55rj/overvie...

There are no comments on this comment yet.