It is just an unfortunate circumstance of the community not efficiently converging on well designed standards quickly enough, and so, the rocky soil has sprouted strange twisty crops. There are articles on how the setuptools/distutils frankenhack pretty much set off the whole trainwreck that you see today, e.g.: http://blog.startifact.com/posts/older/a-history-of-python-p...
Armin (the OP) has also written about this sad history previously: http://lucumr.pocoo.org/2012/6/22/hate-hate-hate-everywhere/
The problem with package managers is the same as npm/gem.
..yum.
[1]: Somehow, all my lines aligned perfectly in this comment box at its default size! https://www.dropbox.com/s/slq8x0rqkj7ydzn/aligned.png
Put like that the reliable solution would be a locally replicated SQL database. Perhaps the PostgreSQL or SQLite would be a better target for development than the underlying OS?
Apologies for acting the cynic, but this is the kind of uncertainty that comes when there is constant reinvention, instead of improvement of existing tools.
Actually it is even true for music instruments: A piano is very complex inside and can be played relatively easily, while a violin is much more simple in the fabric but much harder to play.
This is a solved problem with a really bad case of "Not-Invented-here" in the python community.
What's funny is I'm pretty sure gems/bundler and npm/packages were heavily influenced by pythons plethora of crappy attempts (eggs, distutils...etc) at package management.
Anyone who's worked with all three of those would tell you that this isn't yet a “rest on your laurels” situation.
There are improvements to be made to all of them but gosh python is a complete mess in this regard.
Every solution has drawbacks, but python's solution has almost every drawback.
Another illustration, in reverse: the Chinese chopsticks are among the simplest tool you can imagine, and it requires a somewhat steep learning curve, but once mastered it is really the most powerful food grabbing tool ever invented.
The Chinese brush is of the same nature: extremely simple, very hard to master, and very powerful (expressive) once mastered. Just like the violin (and VIM, and git, etc.)
On the other side of the spectrum you have the synth, IDEs, Google search, etc. Simple UIs, simple to learn, but internals are very complex.
First of all, there's the cases in which reality fails to live up to expectations, like this packaging debacle. If there was an obvious and best way to do everything, we wouldn't need to spend so much money and time engineering software. We wouldn't spend so much time arguing about distributed vs. centralized, OOP vs. declarative vs. functional, type systems, build/test systems, etc. The mantra is arrogant on its face.
Secondly, it flies in the face of innovation. It works against building a creative community of users. So often, when Python users point out the holes in the ecosystem, they are simply told "well the long and outdated PEP 5991834 already establishes the obvious way to do it" or "just shim it on top of this inadequate corner of the standard lib" or the use case itself is questioned, which patronizes the criticizer. The mantra becomes an excuse for reinforcing a hive mind philosophy.
In this case, I wonder if it has subtly undermined the development of a diverse and widely supported package index. The whole point of such a package system is to support the belief that there are actually many ways to do the same thing, and some may be better for certain styles/teams/circumstances. Having the opposite outlook as a design principle for a language will attract users less inclined to contribute to or build such a diverse packaging system. I wonder this particularly because the more "laissez-faire" languages like Perl, Ruby, and JavaScript are precisely the ones that have such great packaging systems and ecosystems.
And yes, one usually installs easy_install to install pip, but I don't see how that can create a "very long dependency chain". Even when both tools are installed, you can use either to install packages.