I wrote my master thesis on General purpose GPU programming. The only two reasons you want to code for the GPU is that it can be much faster for some workloads (basically identical computation which can be trivially parralized) and (in some cases) because it can use much less power.

In the particular case of matrix multiplications, lots of problems can be described as matrices (say googles website ranking) and doing calculations on huge matrices is a thing the GPU is great at. That said it is unlikely to be something most people want.