No, there is an equilibrium point. Firstly, because as you point out, spending on military means not spending on other things. And secondly, because it takes a lot more military power to launch a successful invasion than to mount a defense, if you're not planning to invade you can usually spend a bit less than your potential invaders. Big defensive alliances can help with that as well. In a world where everyone was sufficiently uninterested or unworried about wars being instigated, military spending would generally decrease over time (and it generally has). While there are some parts of the world that feel they would benefit from mounting an invasion, for ideological or economic reasons, then some level of military spending will occur.