News for hackers. Stuff that matters.
Re:Limits on Matrix Multiplication by stochastic_monk on Saturday September 01, 2018 @08:08AM (#17891453)
If floating point values were real numbers, simple matrix multiplications would not produce nonlinearities.

However, researchers at OpenAI demonstrated that the use of subnormal floating point values and their discontinuity [0] provides sufficient nonlinearity to learn nonlinear associations.

[0] https://blog.openai.com/nonlinear-computation-in-linear-netw...

Re:Limits on Matrix Multiplication by crankylinuxuser on Saturday September 01, 2018 @12:36PM (#17892619)
Then, would it be better to store rationals as 2 ints with understood division, and irrationals as 2 ints as understood powers?

Why are we working with these numbers so imprecisely, and with such discontinuity? When I've done math work in school, I always worked with the primitives without approximating.. 2^.5 was handled as such unless asked for an approximate answer.

I also ran across this in AutoCAD. Because they force arbitrary precision, one cannot define a sqrt length. 1.414 doesn't reach all the way on a r-triangle with 1,1 .

Re:Limits on Matrix Multiplication by MauranKilom on Saturday September 01, 2018 @03:42PM (#17893583)
That post completely disregards the fact that subnormals are usually orders of magnitude slower to work with for common hardware.