GPS gives you altitude already, no need for a barometer. It's not hugely accurate, but it's more accurate than an uncalibrated altimeter.

but a properly calibrated barometer is WAY more accurate than GPS. 10 years ago I could track a two meters offset (that's the difference between your head and your feet!).

Indeed, but I can't think of any way to calibrate the thing other than looking up your location and pulling down weather data, or simply forcing the user to put in an altimeter setting.