As someone above was mentioning, I don't think that go was ever intended for heavy math programming. It's a great language for writing network applications that is more performant and deployable than scripting languages, yet easier to use than low level languages. Use rust or c.
I do graphics programming and I literally rely on it all the time.