They can but it is a lot easier to remove someone from the community when you can point to actual rules of the community they've broken besides just "we don't like you".
I honestly can't see peoples objections to CoC. If as you say it doesn't actually change anything why do you object?
Because you have to waste time reading it, and because they tend to force people to sugarcoat everything they say so that someone doesn't get "triggered" [1]. They also encourage SJWs (social justice warriors) to waste other people's time with various crap, for example the Node.js pronoun scandal [2].
As if the maintainers/leaders can't remove someone from the community if they wish to, without the need of codes.
P.S. By the way, the Rust Code of Conduct [1] states:
> Remarks that moderators find inappropriate, whether listed in the code of conduct or not, are also not allowed.
[1]: https://www.rust-lang.org/en-US/conduct.html