It could be argued that physics has nothing at all to do with math; merely, that it is a convenient and powerful tool for humans to make sense of it.

I've seen Susskind convincingly (albeit shallowly) argue that more "primitive" animals understand physics and I would be surprised if it was easy to argue that animals understand mathematics without circular logic or quick and loose definitions of what math is.