How many technical users actually check the fingerprint matches the expected one of the server, out-of-band? Almost everyone I know just accepts the unknown fingerprint, so almost nobody knows who the endpoint they're actually connected to is.
In the mid-90's, CA certs were put into Netscape Navigator (IE joined later) in order to facilitate the new wild wacky concept that someone might buy something online. They called it "e-commerce".
Trust was centralized because it was far easier to add the then ~half-dozen CA's rather then somehow vetting every joe that wanted to self-sign their certs. PGP's web-of-trust existed but it was deemed less viable.
Besides, the whole SSL certs thing was a major business premise behind creating Netscape - profits. Without that there might not have been a dot-com and the huge amount of money that followed since then.
Regardless, look at the vast majority of technical users relying on SSH without using a CA for secure communications, had browsers done a better job at self-signed certs we could be doing the same on the web.