My manager's manager told us publicly that whoever sets the goal must also do the work.

Practically he implied that we don't have to agree to any objectives that are imposed on us.