It would be unwise to stop the arms race when your adversaries are proceeding apace in it. There are only two times it makes sense to withdraw from an arms race.

One is when you sign an enforceable treaty with your rivals. That would be something like START.

Two would be when you know that the arms race is very unlikely to bear fruit. For example, it didn't make sense to keep throwing money at the Star Wars (ballistic missile defense) program because it was very unlikely to succeed.

Military applications for AI could change the global balance of power in a fundamental way. If you care about peace and human rights you don't want the repressive regimes to gain an advantage in that sphere. Because that makes conflict more likely, not less.

I'm sorry, but this is a very naive perspective. And completely chronocentric. The "arms race" started when the first hominid learnt how to throw a rock at his fellow species. Do you really think we are simply going to "end" this arms race?

I don't mean to be snarky, but with the inevitable colonization of space, do you really expect people to just be super awesome peaceful and loving? And not have an arms race?