News for hackers. Stuff that matters.
Re:Beating Floating Point at Its Own Game: Posit Arithmetic ... by dnautics on Sunday July 02, 2017 @03:07AM (#14681289)
I'm just going to be blunt here. John and I have decided that we need to be more marketing savvy after he's had trouble with several rounds pitching other floating point formats. Posits are just an intermediate step to try to build acceptance for valids, so there's a lot of effort put into branding.

A couple of points: 2-4x faster means 2x faster in dot product based simd and 4x faster in matrix simd, assuming that your bottleneck is memory throughput.

The sigmoid function realistically isn't a bottleneck in general, but you gotta admit it is pretty cool to have a ~zero clock cycle approximation. (I've tried to rein in John a bit on this one)

Re:Beating Floating Point at Its Own Game: Posit Arithmetic ... by stephencanon on Sunday July 02, 2017 @07:12AM (#14681987)
If you're bound by memory throughput, you can't go beyond a 2x speedup (there's 1/2 as much data to move in an 8b format, whether it's in vectors or matrices doesn't matter). I still don't see any reasonable expectation for 4x.
Re:Beating Floating Point at Its Own Game: Posit Arithmetic ... by dnautics on Tuesday July 04, 2017 @06:37AM (#14694933)
if your matrix contents are static during your rate-limiting step (as they are for most DL applications) your FLOPs scale with O(n^2) relative to your memory throughput on your vector component.

[a b, c d] dot [e, f]

is four multiplies

[a b c, d e f, g h i] dot [j, k, l]

is nine multiplies.