Re:Cerebras Launches Qwen3-235B, Achieving 1,500 Tokens per ...
by xtracto on Wednesday July 23, 2025 @10:13AM
(#44659457)
I wonder if the fact that we use "floating point" is itself a bottleneck that can be improved.
Remembering my CS classes, storing an FP value requires the base and the exponent; that's a design decision. Also remembering some assembler classes, Int arithmetic is way faster than FP.
Could there be a better "representation " for the numbers needed in NN that would provide the accuracy of floating point but provide faster operations? (Maybe even allow to perform required operations as bitwise ops. Kind of like the left/right shifting to double/half ints. )
Re:Cerebras Launches Qwen3-235B, Achieving 1,500 Tokens per ...
by kadushka on Wednesday July 23, 2025 @10:31AM
(#44659642)
Could there be a better
Yes. Look up “block floating point”.
Re:Cerebras Launches Qwen3-235B, Achieving 1,500 Tokens per ...
by llm_nerd on Wednesday July 23, 2025 @10:30AM
(#44659636)
Sure, we have integers of many sizes, fixed point, and floating point, all of which are used in neural networks. Floating points are ideal when the scale of a value can vary tremendously, which is of obvious importance for gradient descent, and then after we can quantize to some fixed size.
A modern processor can do something similar to an integer bit shift about as quickly with a floating point, courtesy of FSCALE instructions and similes. Indeed, modern processors are extremely performant at floating point math.
Re:Cerebras Launches Qwen3-235B, Achieving 1,500 Tokens per ...
by xtracto on Wednesday July 23, 2025 @10:14AM
(#44659471)
Shit I'd love to do R&D on this.
Re:Cerebras Launches Qwen3-235B, Achieving 1,500 Tokens per ...
by h3lp on Wednesday July 23, 2025 @04:22PM
(#44663576)
A common FP4 layout is 1 sign bit, 3 exponent bits, 0 mantissa bits. There's just not that much difference in complexity between that and a 4-bit integer---the ALU can just be a simple table lookup, for both FP and integer.
The smallest real floating point type is FP4.
EDIT: Who knew that correctness is controversial. What a weird place HN has become.