What you are saying only applies to iOS really.
You can install Android on devices that are already running some version of Android. And some fringe cases like the HTC HD2 and HP Touchpad (I had both, the Touchpad didn't work too great). But that's far from what the OP was asking for.
Hardware manufacturers compete with each other for market. If every handset is the same compatible hardware, that just leaves build quality and price as the variables and they are orthogonal.
That means hardware people also like to add features like GPS, compass, different cameras, expansion ports, heartrate monitor, fingerprint recognition, dual-sim, etc. etc. and we're back to drivers and testing.
Dell, the leading hardware supplier lead the race to the bottom and now flounders. PC commodity markets are cut-throat and it is Intel that makes the real $.