Because different solutions have different tradeoffs. It doesn't appear that there is a single solution which is best for all use cases.

So different systems switch between different methods as their prieceved value of the different usecases change which upsets people who see the values of the tradeoffs differently.

If you have the perfect solution I'm all ears.

One issue on Linux in particular is the [relatively] tight bond between your kernel version and libc, which makes using software compiled with different version of libc problematic part of the time (a particular libc version still supports multiple kernels though).

I experienced this lately when my Rust-compiled binary used too modern a libc version for the aging Docker container environment we used for deployment, which forced me to use another Docker container for local development -- which obviously isn't ideal and removes the 100% reproduceability promise.

Because we love re-inventing the wheel instead of repairing the wheel.

fundamental issue between "change everything once in a big step" and "many small changes".

First is pre internet, because updates cost money, and because pre internet security issues weren't common.

Second is now, change one thing every day, always run the latest code, automated testing to make the latest code always work. Also means don't need a security branch and a feature branch.

People hate change, and linux people have the ability to say no and do their own thing.

Because it is a hard problem?

Python has easy_install, pip, anaconda and wheel; virtualenv to isolate packages. Node has npm and yarn (is it resolved?). Ruby gem defines version in code, bundler in Gemfile, Gemfile.lock; vendor/, rvm gemset, BUNDLE_PATH to isolate packages. Even developers can't find right answer.

Because it matters?

Package management is the main differentiator, I love pacman, I love PKGBUILD and makepkg.