Always good. The system built into display settings ("click yes if you can read this, or the change will be reverted in 15 seconds") has saved me a number of times. No reason not to apply that to other settings where the data channel is the same as the control channel.
OpenWRT router distribution has had this for years, it's amazing! (As is OpenWRT)
OpenWRT also has SQM CAKE which saved my sanity on parents DSL connection for years. As far as congestion control and bandwidth sharing goes, nothing else compares
Alternatively some sort of watchdog timer would be a great addition (e.g. rollback within X minutes if the changes are not confirmed).
I’m saying this is the problem — the device configuration approach should be safe by default.
(Which is an issue with the /automation/ defaults. I've learned enough to do commit confirm, but by default ansible does a hard commit.)
Like the post-mortem says, they will put mitigations in place, but this is something every network admin has to implement bespoke after learning the hard way that the default management approach is dangerous.
I’ve personally watched admins make routing changes where any error would cut them off from the device they are managing and prevent them from rolling it back — pretty much what happened here.
What should be the default on every networking device is a two-stage commit where the second stage requires a new TCP connection.
Many devices still rely on “not saving” the configuration, with a power cycle as the rollback to the previous saved state. This is a great way to turn a small outage into a big one.
This style of device management may have been okay for small office routers where you can just walk into the “server closet” to flip the switch. It was okay in the era when device firmware was measured in kilobytes and boot times in single digit seconds.
Globally distributed backbone routers are an entirely different scenario but the manufacturers use the same outdated management concepts!
(I have seen some small improvements in this space, such as devices now keeping a history of config files by default instead of a single current-state file only.)