The problem is mainly that on IRC messages coming from a server connection are assumed to never be lies. A server can do anything from disconnecting people to spoofing messages.

Which is in fact how services such as nickserv operate. If I don't log in to nickserv, nickserv will spoof a nickname change, blocking me from using the registered nickname.

Having server connections be freely available to everyone is worse than giving everyone OPER powers.