- What happens when you start combining waves together to create more complex signals? This is pretty important, since any real instrument produces hundreds of harmonics with complex attack and decay properties, so decomposition of the fundamental frequencies won't be as accurate as with toy examples.
- Following on from that: the effects of aliasing. It definitely exists and I have a very good understanding of aliasing with respect to computer graphics, but what effect does aliasing really have on an audio signal? In CG it's something that's talked about all the time and there are tons of papers about it, but it seems (from the outside at least) that audio guys only ever talk about aliasing in very hand-wavy terms.
- Although we can't hear sounds (much) above 20KHz, we can detect artefacts such as "beats" produced when harmonics are slightly mismatched. Is is possible to show that such information either isn't lost, or that what is lost is either below the noise floor or outside the audible frequency range? This particular one is a fairly common complaint made by audiophiles about 44KHz/Nyquist, so it would be nice to see it addressed head-on.
FWIW, I'm a bit of an audiophile myself, but not one of those people who thinks they can hear a difference between 192kbps MP3 and uncompressed, let alone 16/44 vs 24/192. Generally the noise floor in the original recording is too high to tell the difference, even if you could hear a difference in artificially constructed pathological cases. But I am interested in really understanding what information is lost, what isn't, and why that may or may not make any difference. In other words, what are those pathological cases? This video is a really good start and clears up a lot, but in some ways it only scratches the surface.
What we effectively did was use a pulsed fiber laser, which produces _very_ short pulse widths of light. Connecting this to an electro-optical modulator, we can effectively capture a very fast snapshot of the electrical signal. Use some analog circuitry to hold that level constant for the slower analog-to-digital conversion, and we had an AD converter that while sampling relatively slowly, see's only a tiny portion of the signal for it's actually sample.
What was amazing about this was that the laser pulses were so short, and the 'timing jitter' of the pulses so low that we could cleanly see the expected aliased sampling frequency when saming a sine wave signal at large multiples of our sample frequency (iirc up to the 40's). Eg, inputting a sine wave around 10 GHZ and we'd see only a single sine wave in the sampled signal at it's row sampled frequency with minimal artifacts.
So if you are wondering how aliasing affects audio samping, draw a sine wave and see what happens if you sample it at a much lower frequency. You'll get a lower frequency sine wave. Without filtering the input signal, it's impossible to tell if this is a real low-frequency sine wave or something above Nyquist that has been down sampled.
N.B. If you're wondering why our AD converter was better than a commercial one, we also added optical demultiexing to get an extra order of magnitude improvement in sample rate. This is possible using our electro optical setup due to the laser's low jitter and the ability to hold the signal steady, a purely electronic version will have difficulty with the timing to see much actual improvement in SNR.
See also http://xiph.org/~xiphmont/demo/neil-young.html which was posted here a week ago and received 441 comments. https://news.ycombinator.com/item?id=8689231
When audio is aliased, the high harmonics will wrap around the Nyquist frequency and come back as audible, inharmonic tones. Aliased audio really sounds quite bad, in a way that would make any listener say "oh god make that horrible noise stop".
This is probably why it's easier to recognize the effects of aliasing in video than in audio -- you're not hearing much aliased audio. Aliased video is fairly common and seen as acceptable in some contexts, while anyone who produces aliased audio is going to fix their mistake before inflicting it on listeners.
[1] https://en.wikipedia.org/wiki/Aliasing#Online_audio_example
For instance, you won't notice Moiré patterns in photos very often, but it's easy to construct a test image that demonstrates the problem. The underlying issue of sampling error hasn't gone away in the photo: it just gets lost in the background noise or among smooth-shaded surfaces without hard edges, which makes it harder to see. But maybe now and then you'll have a sharp edge in a real-world image where the effect is noticeable. Depending on how close it is to the pathological case, it might not jump out and make itself immediately obvious.
I mention Moiré patterns because these are (I believe) precisely the same effect as the given audio aliasing example. Moiré patterns aren't a form of sampling error per se, but they can be caused by uniform sampling (which is how digital quantisation typically works for both audio and images).
Why? What do you mean? What scale? What do you mean by distortion? The only only only thing sampling does is move frequencies beyond the nyquest frequency into the sampled signal by taking the signal at f, and sending it to f mod f_n where f_n is the nyquist frequency.
In terms of more complex signals, or frequency "beats", the same principles apply. If two high frequency (above-20kHz) signals interact in such a way to create an audible (below-20kHz) beating, the beating will be accurately recorded by the ADC.
So the rule of thumb is that each bit gives -6dB more noise floor. This gives us a rough digital noise floor of -96dB for 16 bit or -144dB for 24 bit. But you can add dither to randomize the digital noise, and turn it into "white noise" or the like.
Aliasing is taken care of by the "reconstruction filter" ( named for the "reconstruction" part of the Shannon-Nyquist theorem). This has been a solved problem for some time now. There's a super-high speed digital filter in the DAC phase that can be measured to eliminate all aliasing products. Implementation quality varies, but even cheap stuff now does a good job of it.
Yes; it's unusual to have a room for recording that gets to -70dB much less -96dB. -40 to -50 is much more typical for, say acoustic guitar. Noise doesn't add like you'd think; the dominant noise wins, basically.
Combined waveforms make for another waveform. They add brilliantly and this isn't lossy at all. There are pathological cases, but they're not likely found in nature.
Transducers - headphones, speakers, microphones - all impart much more error than the electronics in the middle.
Aliasing is a linear effect. If L is an operator that aliases at some frequency, L(Aa(t) + Bb(t))=AL(a(t))+BL(b(t)). Aliasing is in fact multiplicative! Any time you wish to understand aliasing, you really want write the signal multiplied by a dirac comb (a train of equally spaced infinitely narrow pulses). If you wish to understand nonfinite sampling widths, it's effectively convolution with the shape of sampling pulse... which looks like low-pass filtering. Ya dig? This is the frequency on your scope that's stupidly above the sampling rate. Why's my scope have a bazillion ghz bandwidth when it only samples at 4 samples oer week? It's because it's real good at getting you that signal, aliased down into the sampled signal.
> what effect does aliasing really have on an audio signal
PRECISELY the same effect it does on video, except in one dimension, and you haven't understood aliasing until you've understood it in one dimension.
> Is is possible to show that such information either isn't lost
It's lost. Completely... if done properly. First you need to understand Whittaker-shannon interpolation. The result of which is that a CONTINUOUS BAND LIMITED signal is PERFECTLY represented by a train of discrete samples. This is WEIRD. Really weird. Prove it to yourself. It's VERY important in sampling. Say you then perfectly sample a signal with harmonic content ALL over the place. EVERYTHING gets mirrored down into your view, if the dirac comb has infinitely narrow pulses. So at the least you lose frequency information! You only know the original frequencies modulo the nyquist frequency! If you sample CORRECTLY, you band pass filter your incoming signal so there's NOTHING above the nyquist frequency, then you have a perfect digital representation of your original signal.
If you wanna understand signal processing, you really can't use your intuition. Practice understanding things in the frequency domain and time domain, and generally there's only one proper basis to understand a phenomenon. Filtering is best understood in the frequency domain, for instance. If you try to understand filtering in the time domain, you might begin to think that it bounds the speed at which the signal can vary but that's not a useful intuition. Aliasing is best understood in the frequency domain, since it moves stuff down in frequency mod the nyquist frequency. If you try to build an intuition in the time domain, you'll think that there's some signal lost when you multiply by zero between pulses... and you'll guess at what it was and think "distortion" or whatever but what's lost is just WHERE each frequency is... so be very careful about what basis you choose!
If lowering the number bits still produces the same output wave as input wave what and where is the lower limit that changes the sound?
Finally how is this 24 bit 16 bit related to my 128kbps mp3s?
If you don't dither, then the quantization error will likely be something harmonic rather than the hiss you hear in that video. This is much more noticeable[1].
Lowering the number of bits produces the same wave, but with more error. Dithering ensures that the error is random noise, which is why you hear "tape hiss" in that video as the bits gets lowered. With a tone frequency chosen to maximize the harmonic quantization error and no dithering you would start to hear it.
Lastly, 24 and 16 bit are largely unrelated to mp3s. In practice, mp3s are decoded to 16 bit audio, but they don't store the audio as PCM, but (roughly) rather as frequency/amplitude pairs which could in theory be decoded at any sample size. In any event, the absolute magnitude of error introduced by encoding a 128kbps mp3 is way, way, way larger than 16 bit quantization error, and the goal of the mp3 encoder is to find a way that is as inaudible as possible.
MP3 is an old format at this point, so it does have audible defects in some types of music at even the maximum bitrate (certain percussive instruments are an issue). Vorbis and AAC work in some ways similar to mp3 but are strict improvements (though for the love of music, don't use the FAAC decoder, it's generally considered horrible).
1: Still not noticeable at 16-bit audio, as a similar test to the one in the article was done with no dithering on the 24 -> 16 bit conversion. at somewhere between 12 to 14 bits it would start to be discernible.
If you dither improperly, then you'll end up with a sine wave plus some noise (less than if you had dithered) plus some harmonic distortion.
When talking about audio, one typically doesn't compare in the time domain, since that's not even remotely how ears work. One compares in the frequency domain, and until your quantization error is very large compared to the signal, there will still be some of the original content in the result.
This isn't a great analogy, as eyes and ears operate quite differently, but it does sort of get to the issue. Look at this image: http://en.wikipedia.org/wiki/Dither#mediaviewer/File:1_bit.p...
If you consider the original to have been an unquantized, but already sampled grayscale image, then essentially all of the pixels have large quantization error (since they were originally some shade of gray, but now they are all either black or white), so in one sense the original image is gone. But in another sense what you see is the original image, plus some noise.
In short, it was seemingly made in-house.
[1] http://xiph.org/video/video.js