I've never even thought of installing more than one package manager. I package CPAN modules when I need them. If I needed gems, I'd probably do the same thing.
I am always weary of installing gems (actually, I use mostly Python, so I install eggs). It seems wrong to go against APT.
I can understand that, but it's easy enough to specify the apt repository as a parameter to the gems command. If you really want it as the default, it's trivial to alias that. There's something to be said for just letting your packages have the default behaviour.
Apart from that though, yeah, repositories are pretty sexy indeed.