The reason for 96kHz or even 192kHz sampling is that your filter can have a much gentler roll-off.
And the studio can master the recording from a high sample rate recording using an enormous FIR filter, because latency and computing power are mostly irrelevant.
High sample rate equipment just allows you to listen to studio intermediate data without having to have it downsampled first.
But there are problems. For example, if there is any non-linearity in the equipment this can easily cause inaudible tones in the ultrasonic range to intermodulate with each other to produce tones that are audible.
Nothing is simple.
Or no?
If you mean "does a 44khz only output a square (or triangle) wave at 22kHz, the answer is "no."
If you have two alternating values at 44kHz, the output will be a perfect (to the limits of noise &c.) sine wave, since the DAC is band limited.
If this is still confusing to you, I recommend watching https://www.youtube.com/watch?v=FG9jemV1T7I
You can only capture all the information if you sample at twice the rate of the highest frequency of interest.
If it would, it wouldn't be a sine wave anymore, and you could decompose it into its component of phase and amplitude shifted sine waves, which will occupy all kinds of places in the frequency domain, many of them higher than your initial 22 kHz.
That's in fact what we mean when we say "a 22 kHz signal": An ensemble of all of these overlapping waves represented as a single signal. If you're sampling that with an ADC, you're not only sampling the the 22 kHz wave, but you're also sampling it – and that, but only that, can in fact be described with a single bit per ADC readout.
Looking at it from an information theoretical point of view, the Kolgomorov complexity of "a sine wave of 22 kHz and amplitude x" is pretty minimal – just from that sentence, you can perfectly recreate that signal with no sampling whatsoever, and one ADC readout tells you its amplitude and phase. It can carry a bandwidth of 0 Hz.
This reaches the edge of my knowledge, but I think this is basically because the discrete values are 'slewed' (aka integrated aka low-pass filtered) into a continuous signal by the DAC.
See: https://en.wikipedia.org/wiki/Sampling_(signal_processing)
a_i sin(2pitf)
where each a_i is the amount of a given frequency present. If every wave sample you have goes [1,-1,1,-1,1,-1...] then 100% of your wave is accounted for as a sin wave at the sampling rate. There is zero additional frequencies present with unknown amounts. If your wave samples went like [0,1,0,-1,0,1,0,-1...] the only frequency of sound present in the signal is the one at half the sampling rate. The whole wave is accounted for.
On a more physical level, its ok if your sample only sees +1/-1 every other sample, because when that electrical signal turns into pushing a magnet the magnet has to accelerate smoothly in accordance to Newtons laws. In other words, flipping the voltage between +-1 as a 40khz square wave becomes a perfect 20khz sine wave when it reaches the speaker.
20hz-20khz is the human range, more or less. For most adults it's more like 15khz.
The 44.1khz sampling rate of CD audio was chosen because it can represent frequencies up to 22.05hz (44.1khz / 2) which safely covers the frequency range of human hearing with a bit of headroom for good measure.