Believe it or not, I have never played any of the Primes.
I wish I was as eloquent as the OP in describing the experience, since I would love to have that feeling again. Oh well.
The circumstances of a good game well played can burn in and amplify the experience. So I consider myself lucky, since quite few games feel that much deeper given the state of my life while playing them. I can see and agree with all sorts of flaws in the games, but they'll remain special simply because at the time, they were.
I don't really have much to add to that, except that this is a really succinct way of describing, I think, what you, the author, and I are trying to say.
http://www.polygon.com/2015/1/29/7946129/metroid-prime-trilo...
The original Metroid needed them for expensive loading transitions between areas, as palettes and the scroll buffer were swapped out. Interestingly, they also serve that purpose in Prime... If you've ever shot a door and had it change color to show it unlatched but refuse to open for several seconds, that's the console stalling on loading the area behind the door.
But especially in the two Primes, it feels contrived. I absolutely adored the two games, but there's a feeling the whole thing is a setup for the character. Too many special circumstances in the ice or rock that somehow awaited Samus showing up at just the right time with the right weapon. The earlier games don't have that, I think, and I can see how Super Metroid starts the trend.