News for hackers. Stuff that matters.
Re:Show HN: Sound visualisation, better than FFT (iOS) by avaku on Friday April 17, 2020 @11:58AM (#22900125)
I also thought this might be useful for people who have problems perceiving some frequencies (maybe all?), to just see the sound. I am not sure what is the correct way to call people who have trouble hearing, but I am really wondering if this could help someone, and I'd like to help make it better. (edit: typos)
Re:Show HN: Sound visualisation, better than FFT (iOS) by andai on Friday April 17, 2020 @12:19PM (#22900319)
I had the same idea. I was looking at some spectrograms of a podcast in Audacity and I began to recognize different people's voices and frequently used words. So I realized that with enough effort it would be possible to learn to read in this way.

I also like looking at spectrograms of birdsong. I had a setup where I would see the spectrogram of sound coming through my microphone in realtime. I would place it on the windowsill and a few times I would see very faint birdsong I hadn't noticed with my ears, and then I would "tune in" my ears to that frequency range and I would hear it.

Thinking about it the ideal form factor for a realtime spectrogram for the hard of hearing (or the curious) would be something like Google Glass. (Originally thought of a smartwatch but that doesn't make much sense for an additional sense).

There's a project that's the opposite of this where a colorblind man attached a sensor to his head that converts color into sine waves he can hear. Adam Montandon. Eventually this electronic sense became a natural part of his experience and he began to dream in "color".

Re:Show HN: Sound visualisation, better than FFT (iOS) by avaku on Friday April 17, 2020 @12:22PM (#22900345)
I am not sure it would be possible to visually recognise speech, but might help people with hearing difficulties to perceive a concert better or something.

Funny think about birds! Yesterday, as I was preparing to release the app, birds were singing outside of my window (coronavirus lockdown, so quiet), and I noticed how fun to look at their songs on my app! I've spent literally 30 minutes looking at that :) I've posted a link to birdwatchers reddit :)

Re:Show HN: Sound visualisation, better than FFT (iOS) by avaku on Friday April 17, 2020 @12:25PM (#22900371)
On the topic of recognition though, I think it would be much easier to implement speech recognition using the core algorithm used here, rather than from FFT for example. I'm only at the beginning of trying to use the frequency detection algo for something else apart from visualisation... The difficulty is that the dat it generates is much more detailed for higher frequencies, so it's hard to adapt modern ML algorithm inputs to it, I need to think more...
Re:Show HN: Sound visualisation, better than FFT (iOS) by avaku on Friday April 17, 2020 @12:27PM (#22900392)
Interesting idea about transforming colour to sound! P.S. I am red-green colourblind too :) Although, I don't find it a problem.
Re:Show HN: Sound visualisation, better than FFT (iOS) by nmfisher on Saturday April 18, 2020 @12:15PM (#22908690)
One of my favourite (electronic music) producers mentioned recently that he has been developing tinnitus.

Normally this would affect his ability to hear certain frequencies, but he’s so accustomed to working with raw waveforms, he’s not too concerned. He knows exactly what the sound he wants should look like.

Re:Show HN: Sound visualisation, better than FFT (iOS) by avaku on Saturday April 18, 2020 @12:44PM (#22908915)
Thanks for comment. I think this can be useful to music producers (in some way), but the market for plugins is so tough, basically hard to survive even for one developer on revenue from one plugin. I’ll be thinking about this.