My biggest problem is that figuring this state of affairs out is like pulling teeth.
Besides, there are other ways to get that flexibility. CPUs for instance have a very flexible ISA: even if they’re set in stone, we can use them for pretty much anything, the only real limitation being performance.
Now when I think about it, even if firmware can significantly change the functionality of a chip, I don’t care too much of it being proprietary. Not more than I care about fixed hardware being proprietary. It’s okay for a piece of hardware to have 3 or 10 different configurations, as long as (i) I can trust that each configuration works as advertised, and (ii) the ISA I got from each configuration is publicly documented.
There’s a practical reason for this wish: hardware is typically orders of magnitude more reliable than software. That’s mostly because buggy software is easily updated, while flawed hardware is often impossible to sell. CPU bugs do happen, but they’re sufficiently few and far between that when my programs behave unexpectedly, I can safely assume it’s the software’s fault.
I don’t want drivers any more, I want the hardware’s manual. https://caseymuratori.com/blog_0031
Personally, I don't care much about firmware being proprietary, if it cannot be used to meaningfully change the functionality of the chip. But I do care about the chip ultimately following a public specification. I want an ISA just like x86, ARM, or RISC-V. Tell me the size and format of the ring buffer I must write to or read from, and I'll program the rest.
But even that is still too much to ask for apparently. Graphics cards are getting closer with lower level drivers like DX12 and Vulkan, but we're not quite there yet.