I'm not generally sold on the "worthy successor" or "human disempowerment would be good" arguments (seems like jumping straight to Hard Mode of alignment when we've not even read the tutorial) but this is one of the ways it might be best for the AI to just… disregard what we say and do something else instead.