Since there's a fixed level of sensor noise, when you increase the analog gain you decrease the level of noise in the shadows relative to the sensor noise, essentially increasing SNR. This isn't true for all cameras, however.
This chart [http://www.photonstophotos.net/Charts/PDR_Shadow.htm#Canon%2... ] demonstrates the difference between three cameras, one of which is known to be largely "ISO invariant" (the Nikon D750), one which is not (the Canon 6D), and one which looks like it has a 'native' ISO of 800 (the Sony A7R II)
As you can see, the D750 doesn't gain much (heh) from increasing ISO -- the total image noise in the shadows isn't improved by increasing the ISO through the range. If you're shooting a 6D in a low-light situation, however, you see improvements in shadow quality up to about ISO 3200, after which it turns out that you will be fine if you just raise the shadows in post. If you're shooting an A7R II you might as well just leave the ISO dial on 800 all the time.
That's not all, though, since it turns out when you increase ISO you basically lose 1EV of highlight off the top of your image, as you allude to, and you also lose around 1EV of DR. (Chart here: http://www.photonstophotos.net/Charts/PDR.htm#Canon%20EOS%20...)
It is, like all things in life, a balance.
This will change in the next few years.
If you're the kind of guy who wants to read the relatively dry writings of someone who spends a whole lot of time figuring this stuff out, the following link might interest you: http://clarkvision.com/articles/digital.sensor.performance.s...