In my experience, the brutality grows rapidly with the number of users. If it's just for you, and you have at least one alternative account, then it's not brutal at all.

A mailserver administrator is a sysadmin. Being a sysadmin is on the face of it unrewarding - nobody pats you on the back when everything is working normally. They only call you when there's a problem (and it's usually urgent, and sometimes critical, and they'd like to know who to blame). So if you run a mailserver with users, it starts to become a people job, and things begin to matter.

People certainly rely on email to a greater extent than other services - even mobile. I can order goods online without surrendering my mobile number, but I always have to provide an email address before I can complete the order.

But I've still enjoyed administering mailservers. Perhaps I liked the slight paranoia induced by trying to reconcile the boss's demands for functionality with his demands for security.