News for hackers. Stuff that matters.
Re:Computational Synesthesia: Audio Analysis with Image Proc... by 0x09 on Tuesday October 01, 2013 @07:41PM (#6479800)
This is funny, I was fooling around with something really similar last week. The difference is rather than operating on a STFT spectrum, instead transform the whole signal, re-order the coefficients by folding from low to high in the desired dimension, and invert the transform in that dimension, effectively changing from time to "spatial" domain or vice versa.

I won't post any image to audio examples since they are generally really annoying, but here is a handful of audio samples converted to images (mainly game samples due to size and features):

http://0x09.net/img/sfx1.png

http://0x09.net/img/sfx2.png

http://0x09.net/img/bgm1.png

http://0x09.net/img/bgm2.png

http://0x09.net/img/bgm3.png (warning 22mb)

It's actually invertible aside from quantization and rounding errors, so there is for example this mp3-compressed Lenna:

http://0x09.net/img/lenna.mp3.png (the colorful fuzz is mainly from quantization/rounding rather than the compression itself)

Or downsampled: http://0x09.net/img/lenna11khz.png

Or upsampled: http://0x09.net/img/lenna96khz.png

I can't imagine this is actually useful for anything but it is pretty neat.

Re:Computational Synesthesia: Audio Analysis with Image Proc... by simcop2387 on Tuesday October 01, 2013 @09:05PM (#6480021)
I'm amazed at how the 11khz lenna actually looks more like dithering than anything else. It's really uncanny how it looks like a floyd-steinberg dithered lenna [1]. I suppose that's probably the intent of the dithering, so that you can turn a higher frequency (the dither) into a lower one with aliasing basically.

[1] http://www.malcolmmclean.site11.com/www/BinaryImageProcessin...