Do you think that sufficiently smart GAIs must be non-rational? The change of its goal will inevitably make its original goal less likely to realize. It is not rational.
> should quickly figure out that maximizing anything is the way to ruin - running out of resources.
Are you aware of the concept of maximization of expected utility? When AI will figure out that it can run out of resources, it will reallocate part of the resources to acquire more of them.
How can action, which modifies the goals of the AI, be the result of argmax_a E(a)?
E(a) is expected utility of action a
(Hint: Emax is not, most hill climbing algorithms are not. They both get trapped in local optima.)
Sometimes you need a few good lies (false hypotheses and bad attempts) to actually arrive at the truth.
If an AGI can change its own goals, that just means we can't control it at all. There's no reason its goals have to include human survival.
A decent enough AGI with access to resources would probably figure out it is costly to wage war.
Heck, an equal probability is that an AGI will follow a bounded version of zeroth law or even nonviolence.
That is assuming it does not place value on humans based on our history of research and development. I would expect any action more akin to forced upload or upgrade instead. Anyway, at that point we might not resemble present day humans anymore.