The US has all sorts of odd subsidies and issues of regulatory capture like this, I'm thinking also the corn subsidies, and the block on buying cars direct from retailers. It's ironic that all of that happens despite all of the vastly stronger rhetoric about cutting regulations. In my opinion it's another indication that a 'starve the beast' approach to government is not effective even on its own terms. To actually get rid of specific bad policy you have to campaign against those measures specifically, and have people in government who are acting in your interests, you can't just rely on a general anti-government government.
This happens despite Norwegians generally being positive to government and regulations while Americans seem knee-jerk anti anything government.
To get simple and good regulation I think you need an effective government and a belief in government. In Norway you can enact large and broad reforms and regulations, while in the US everything seems peacemeal.
Take welfare or healthcare. The US has a huge patchwork of small programs and minor regulations. This is due in part because of a reluctance towards broad but simple government programs like single payer.
A government can set standards for data exchange and bills e.g. which is why prescription can be sent from my doctor electronically to all pharmacies in Norway.
Yes. This is non-negligible part of the problem. There's lots of regulation and that plays right into the hand of the incumbents (utility companies) who can use that regulation to make it harder to compete.
California in particular had a relatively quick plummet in power demand that actually caused older parts of the grid to start breaking. PG&E despite having used negligence to set a bombs with gas line negligence and kill dozens of dogs with electrified ground plates argued they couldn't cut into profitability, and started forcing people to reconnect to the grid even if the customer would exceed the maximum buyback cap PG&E offers (as I understand it, not a homeowner so this specific part is only my reading).
Result is protectionist nonsense that greatly reduces the profit margin of solar for individual installs.