Math definitely has a grammar. In upper level classes and in papers mathematicians even use correct punctuation.

It's all very much just an extension of human language, just like you said.