The most frustrating thing with PowerToys is trying to remap keys (like caps lock to Ctrl). It feels like it's done by intercepting the keypress at runtime in the app rather than being configured at the system level, so if you happen to, say, hit your new Ctrl key when the CPU is pegged, it'll revert back to caps lock and then also get stuck. So you have to go into PowerToys to unbind the key, turn off caps lock, then rebind it.

There's another app that does this in the registry I think, but I keep forgetting the name of it.