Because your users cannot be trusted. I run servers which coworkers, all developers, get access to. It's either ssh keys or 2 factor authentication because most don't give a fuck about security and will insist using garbage passwords if not prevented from doing so (and then they will complain), even though the use of a password manager is mandatory.