And in my specific case, the group with the singing keys is also the group paid to tell us "no" whenever a release is blocked by process reasons.
I got dragged into defending my technical solution, but my point was that if the ability isn’t needed, don’t have it. You can break the glass when when you need to. Good process make deviation from it more visible. Our code signing keys belong to a team we already need wet ink signatures from to release software.
I can go into the biohazard labs with shorts on easier than I can leverage our technical disaster recovery. I’ve only ever had to do the latter.
You know that backup system your production servers have?
That also has perfect _write access_ to everything, and it's needed so _restoring_ after something goes wrong works.
What? Why?
Nobody on my development team has access to the production code signing keys. And nobody - at all - has the ability to remotely make a production system take an unsigned update.