The problem is actually the opposite: The hardware vendors are providing the firmware instead of a spec that allows you to write your own. Then you're stuck using their software to use their hardware, even if it's awful.

It's perfectly fine to provide a reference implementation, but that's not a substitute for hardware documentation sufficient to let you make your own if you don't want the vertical integration.

Because the party with the economic incentive to make it better is the end user. Not all of them have the capacity to do that, but only one of them needs to and you have better code available to the whole world.