I think a more helpfully formulated question would be "why would I be OK with an AI government that caters to these objectives?"

the AI can prove that it catered to those objectives, isn't that the problem you stated?

The AI is unaccountable. If the leaders get good ideas from the AI, then cool, let them present them and take responsibility.

accountability does not guarantee future improvement, while an AI can be continuously retrained

AI does not guarantee future improvement either. Humans can also be continuously retrained. I don't understand your point.