I don't think it is possible to make Windows installers fully transactional, but architecture astronauts from Microsoft did their best to pretend that MSI is actually capable of transacted installs. Which is why MSI is such a horrorshow.
Aside of the (slightly) transactional support, MSIs also allow for easy centralised deployment and because correctly authored MSIs are declarative only, an admin can potentially audit the installation process before running it.
Finally, when you kill dpkg or rpm at the right time, you will have an equally non-working install under your Linux distro as you will have a non-working install of your MSI file when you kill msiexec at the right time.
Don't get me wrong. I agree with you that MSI is suffering from overengineering and I agree that the UI is really messy, especially with regards to updates, but instead of trying to find different, less standardized installers, we should invest in better tooling (compared to WiX) and maybe tell Microsoft what we need changed.
[0]: https://msdn.microsoft.com/en-us/library/windows/desktop/hh8...
https://github.com/geary/jklmouse/blob/master/AutoHotkey/Sou...
Or were you saying that your installer runs elevated and installs to Program Files, and also wants to write some files to the user profile? I would think that should just work?
Shameless plug: JKLmouse gives you home row mouse control in nearly every Windows app. It's like MouseKeys but doesn't require a numeric keypad, so it works on laptop keyboards. Not too many people know about it; I wrote it for my own use years ago (first in C, then later in AutoHotkey). It is so nice to be able to move the mouse pointer pixel by pixel.
NSIS and Inno Setup are great when you want great control of your installation process instead of following the standard Microsoft Installer (MSI) idiosyncrasies which can give non experts a lot of headaches.
It is important to note that NSIS is not transactional like MSIs and is not the official Microsoft supported installation mechanism in [big] organizations.