> One interesting idea that came to me last night was trying to reproduce the physical 3D model of an instrument based on its spectrographic fingerprint.

In case you haven't already read it, this might be of interest to you:

Mark Kac: "Can One Hear the Shape of a Drum?"

https://en.wikipedia.org/wiki/Hearing_the_shape_of_a_drum

https://www.maa.org/sites/default/files/pdf/upload_library/2...

Would be interested to see this applied to drums that have intra-beat tuning, such as the talking drum or the idakka.

> Spectograms of each note will also make it strikingly clear that there is a dominant frequency invoked and overtones (harmonics)

It's not even about which frequency is dominant. I'm not an expert on psychoacoustics from the "sciency" side, but basically our brains can figure out the fundamental note based on the harmonics alone, even if we don't hear the fundamental frequency itself. De-emphasizing the fundamental frequency is sometimes used in synthesis just to create an interesting sound, but most notably this phenomenon is heavily relied on in modern heavy bass electronic music and down-tuned metal. A really common trick is to distort the bass track, so it can be still heard even if speakers can't reproduce that frequency, as distortion generates more harmonics. I don't mean distortion in literal sense, but in audio effect sense, so effects like saturation, overdrive, fuzz, clipping, etc.

Being able to “hear” a fundamental when it’s not there in a spectrogram but the upper harmonics are is known as the missing fundamental and has been known and studied for over 100 years. It seems likely that the auditory system is not only doing a spectral breakdown of sounds (like an FFT) but also something like an autocorrelation of the signal. That autocorrelation will recover the fundamental as long as enough harmonics are present

> but basically our brains can figure out the fundamental note based on the harmonics alone, even if we don't hear the fundamental frequency itself

I guess this is valid for trained musicians. For me, I can't possibly identify the same note in different instruments, as all these sound very different to me.

The only thing I seem to be able to do is to identify a song based on the very first second or two. Because they are different sounds, not just notes.

Very interesting, since the artificial “ghost” note is at a lower frequency, would they essentially become an “undertone” ?

Undertone is something different, but I never explored that concept and what could it be useful for, so I can't really explain it

Not an undertone, it's fundamental frequency. Harmonics/overtones follow a very specific pattern:

https://en.wikipedia.org/wiki/Harmonic_series_(music)

...so in most cases there isn't really any ambiguity on what the actual fundamental frequency is. The best way to demonstrate it is probably with a high-pass filter (aka. low-cut filter). High-pass filter allows only higher frequencies to pass through, or in other words it cuts out lower frequencies. On the example below you can hear that as the lower frequencies are getting removed from the signal, it's still the same note:

https://www.youtube.com/watch?v=50lRE2Bgag0

I wish the filter sweeps were slower, but that's the best example I could find

No, not really.

The fundamental is always the lowest frequency. All overtones are some (almost always integer, or _nearly_ integer) multiple of that.

I read somewhere about people doing this with samples of thunder, to recreate the shape of the lightning bolt. So cool!

ignorant/lazy question:

I'm a guitar player who took a few months of trumpet lessons when I was a kid. I recall that you can produce several different notes with the same fingering on the trumpet depending on the shape of your mouth. is this similar to the natural harmonics you can produce with a guitar by covering (but not fretting) the strings at certain nodes?

Brass player here with a few months' worth of guitar ability. You're correct about the harmonics being at play, but the mechanism's a bit different.

To elaborate/review, your lips are the guitar strings in this equation, and naturally will behave much like a string on a chamber instrument or fretless guitar (as is obvious from "free buzzing" and mouthpiece buzzing). The length of the tubing then dictates which frequencies will resonate with your lips, meaning your lips will want to "settle" into something in harmonic resonance with that length of tubing.

The key difference here is that your lips themselves are basically "fretted" to the harmonics (though with practice you can bend that quite a bit), since the corners of your lips are moving (slightly; it's a short string!) in and out to go higher or lower (respectively), and since they'll want to vibrate at a harmonic (the vibrating metal and air impart a force on your lips for the same reason your vibrating lips impart a force on the metal and air, so it takes much more effort to buzz against that harmonic than it does to just ride it and keep that feedback loop going). Further, the mouthpiece itself is basically a capo in this context, so your lips are always "fretted" to the mouthpiece's constraints (this is a big part of the reason - if not the entirety of it - why trumpets have tiny mouthpieces and tubas have giant mouthpieces).

I suspect that if you were to replace the body of an acoustic guitar with a really long pipe, you'd see/hear similar dynamics at play: a string tuned or fretted to one of that pipe's harmonics will keep vibrating for a good while, and a string tuned/fretted to something else would stop vibrating sooner as the destructive interference sucks energy out of it. And the former would likely be much more audible than the latter.

I know HN frowns upon such meta comments, but thank you for this comment! Wonderfully clear exposition - I followed along completely with your comment and now have some level of understanding of harmonics and brass instruments where I had none before. This was a good day, I learned something today that I have long been curious about!

Your disclaimer "I know HN frowns upon such meta comments, but " was the only thing wrong w your comment. (Which I upvoted). As a guitar player and former trumpet player, I dug this tangent too.

Just to reiterate the other response to your comment, meta comments are often welcomed, especially when they are expressing thanks or gratitude.

Content-less comments that derail the conversation, comments that are needlessly inflammatory - these are the kinds of meta comments that are not wanted.

I don't play trumpet, but I think that is basically what is going on. You can do pinch harmonics on guitar, which silence certain harmonics or even the fundamental while retaining others. It sounds like changing mouth shape does a similar thing on trumpet.

edit: here's a fantastic video on pinch harmonics: https://www.youtube.com/watch?v=eTWxCdoyol0

Yes, that's exactly right. "Covering but not fretting" a string on the guitar will dampen the lowest harmonic because the full length of the string is unavailable for vibration [1]; frequency is one over wavelength (e.g. twice the frequency is one half the wavelength). You can pluck either side of the string when covering it to produce the same harmonic (neck side or body side).

On a trumpet, the embouchure will affect the frequency of the vibration of the air compressed in the tube, and simply drop out lower harmonics, as can be confirmed via spectrogram.

The same thing happens on a guitar, leetcrew I encourage you to try the spectrogram linked at that site with your guitar to note the effect [0].

[0] https://musiclab.chromeexperiments.com/Spectrogram [1] I am under the impression that the fundamental tone of the string requires the whole string-length to resonate, and if it is clamped, pinched, or otherwise muted all you will hear are resonant harmonics that can exist on smaller string segment lengths.

We need to compare this description with yellowapple above.

One of these is true but I’m not clear on whether this means actually both of them are true?

Yes, to first oder, a trumpet is a long tube with a standing wave, which works conceptually like a string (but with different boundary conditions). It's probably the player's imposed frequency hitting multiples of the lowest resonance frequency that leads to different tones.

Great video. In case it's not obvious, the different notes are purely a function of pick/thumb placement. The guitarist is not changing frets, but he seems unable to resist throwing in some vibrato :)

Yeah, on a trumpet you can play multiple notes with the same fingering. The set of notes you can play with a fixed fingering comes from the harmonic series. A bugle is essentially a trumpet with a fixed length of tubing, and famous bugle calls like Reveille and Taps all come from that harmonic series (and can also be played on a trumpet without varying the valves).

My understanding of how this works is that the length of the trumpet's tubing (at any given fingering) permits the air to resonate in standing waves only at one of the frequencies in a harmonic series. The player's lips can vibrate at any frequency on their own, but the big column of air inside the trumpet will essentially lock the column of air into vibrating at the closest frequency in that set.

And this is why we can synthesize different instruments electronically. Reproduce the overtone patterns and you hear the same instrument.

Not to mention, two different pianos or two different violins can sound very different.

The sustained overtones are only half the battle. Getting the attack correct (the sound profile of the first ~10 milliseconds) is really important for differentiating instruments. Plucking a guitar string and hammering a piano string have very different attack characteristics. A flute has a distinctively "breathy" attack.

Many synthesizers use a sampled recording of the actual instrument for the attack, then synthesize the sustained portion of the instrument.

> The sustained overtones are only half the battle.

Actually probably like 10% at most of the battle. My understanding is attack is overwhelmingly dominant in our perception of timbre.

This takes me back to when I played around with sound using the C64 midi. There were four attributes that you could adjust to try and emmulate the timbre of a particular instrument: attack, delay, sustain & release.

Was a lot of fun when I was young.

Are you saying you could say, do a spectral analysis of someone singing or talking, then make a physical instrument that sounds similar?

Sure, you can do that kind of thing in principle. I think the vocal cord model needs to be constrained which will not be very fair, but I was reading papers over a decade ago about simulating speech with physical models of the voice. I assume that's how they guess what dinosaurs might have sounded like.

Of course, with such a complicated and underconstrained system you might need to basically tell it what a human vocal system roughly looks like and let it calculate parameters based on the model. Maybe not though, neural networks are surprising sometimes.