I thought GPUs were involved in all sorts of calcuations like matrix calculations that had nothing to do with graphics.

My understanding is that's not correct. It's possible to do general purpose computing on a GPU with things like CUDA, but your code has to be specifically designed that way and is tied to the drivers/hardware you implement it for. It's not going to automatically happen if you write some matrix code.

I would guess most people aren't doing anything like that. If you are just running a basic site and database you don't need a GPU.