This. Bothered to login to say what you said.

We talk about AI optimized to create paper clips accidentally running amok and turning the world into paper clips. Why isn't profit a similar problem? Government agencies are just AI running on meat. Give them success parameters different than raw profits.

Honestly, why can't we see this as just a variant of the instrumental convergence problem?