News for hackers. Stuff that matters.
Re:Beating Floating Point at Its Own Game: Posit Arithmetic ... by dnautics on Tuesday July 04, 2017 @06:37AM (#14694933)
if your matrix contents are static during your rate-limiting step (as they are for most DL applications) your FLOPs scale with O(n^2) relative to your memory throughput on your vector component.

[a b, c d] dot [e, f]

is four multiplies

[a b c, d e f, g h i] dot [j, k, l]

is nine multiplies.

There are no comments on this comment yet.