Allowing unsigned drivers to be installed by the general user base seems more anti-user, no?

My naive understanding is that drivers are low-level and thus malicious drivers would have more net negative impact on an unsuspecting non-technical user (relative to a piece of malicious non-driver code).