Experiments with Byte Matrix Multiplication

Re:Experiments with Byte Matrix Multiplication by atq2119 on Friday January 10, 2025 @04:11PM (#42660288)

This doesn't feel like a convincing argument. If you wanted to multiply uint8 * uint8, you'd naturally use an unsigned multiply with a uint16 result. That doesn't overflow either.

I believe a better argument is to appeal to the structure of neural networks. Activation inputs into a matrix multiply come out of a non-linear function, and ReLU is a popular function which causes activation inputs to be unsigned. Weights then need to be signed so that the matrix multiplication can have negative outputs -- without negative outputs, you would lose the non-linearity of ReLU.

[ Parent | on HN ]

Re:Experiments with Byte Matrix Multiplication by dkhudia on Saturday January 11, 2025 @03:22AM (#42664280)

This is true but the instruction already existed and it doesn't support uint16_t accumulation. For the reason you mention, activations are uint8_t and weights are int8_t so it worked out well for neural networks.

[ Parent | on HN ]