News for hackers. Stuff that matters.

A comment on Addition Is All You Need for Energy-Efficient Language Models. Parent comment

Re:Addition Is All You Need for Energy-Efficient Language Mo... by bee_rider on Wednesday October 09, 2024 @11:07AM (#41788694)
Interesting… I guess it must be biased, m*2^ee would leave like half of the limited space wasted, so 1.m*2^ee?

I always wonder with these tiny formats if 0 should even be represented…

[ Parent | on HN ]
Re:Addition Is All You Need for Energy-Efficient Language Mo... by wruza on Wednesday October 09, 2024 @11:22AM (#41788855)
I’m not a binary guy that much, but iirc all floats are 1.m*2^e — “1.” is always there except for subnormals. There’s also SEEE FP4 which is basically +-2^([u?]int3).

https://medium.com/@harrietfiagbor/floating-points-and-deep-...

[ Parent | on HN ]