I think the trick for this is to embrace distros that allow for more aggressive installation and removal of packages, plus different versions of the same package to co-exist. Like NixOS or GuixSD. Then, one doesn't need to keep the whole package tree in sync.