If you create a Python lib, 10000 people will pip install it. You have no control on the env.
If you create a deb package, you will see ppa and it will be installed on many various env. You have no control on the env.
Again, you definitely have no experience in shipping software outside of your bubble.
A lib is not "a web project". A cmd line tools is not either. You still need to debug them. People will run them on windows, linux, mac, bsd, and who knows where. And they will come for you.
Now you can choose to simplify the problem and only support a limited number of env. But I guess I'm quite happy the guys who created apache, ffmpeg didn't force me to only used them on Linux with LOCALE set to accept only ascii and CEST.
For example, here is a library I maintain with 12,000+ installs per month that is tested on 7 different environments every commit using Travis: https://github.com/globalcitizen/php-iban
Don't get me wrong. It's a good thing. Most people I know don't even have unit tests. Having a CI is fantastic. 7 env is more than a lot of people do.
But that doesn't even scratch the surface of the combinations of factors you can get. It can't.
You can't install all the locals, all the lib versions, simulate all network conditions and all user inputs.
You will get strange bugs that your CI didn't take in consideration. And some of them will be very hard to reproduce. That's just a fact.
Now knowing you do have all this setup I just can't understand we still disagree. It's impossible you didn't run in those.
In short throwing your hands up in frustration at the complexity of software is not a solution and gets you nowhere. The way we deal with "infinite ways to fail" is to control the environment. These days, quality projects are expected to version control their environments and conduct test deployments within a representative set of environments using a representative set of configurations.
Docker provides an easy way to do this ("always deploy on <distro>-<os>-<version>"), but it's only one approach. Another free and relatively straightforward place to start getting up to speed would be automating build and test processes with Travis CI for an open source project.
Deploying to any number of tested environments is trivial.