News for hackers. Stuff that matters.

A comment on Riffusion – Stable Diffusion fine-tuned to generate Music. Parent comment

Re:Riffusion – Stable Diffusion fine-tuned to generate Music by aix1 on Friday December 16, 2022 @01:32AM (#34011261)
The idea to apply computer vision algorithms to spectrograms is not new. I don't know who first came up with it, but I first came across it about a decade ago.

I just ran a quick Google Scholar search, and the first result is https://ieeexplore.ieee.org/abstract/document/5672395

This is from 2010. I didn't go looking, but it wouldn't surprise me if the idea is older than that.

[ Parent | on HN ]
Re:Riffusion – Stable Diffusion fine-tuned to generate Music by hecanjog on Friday December 16, 2022 @01:33PM (#34018254)
There were a number of systems designed for composers in the 90s (also continuing through to today) designed for the workflow of converting a sound to a spectrogram, doing visual processing on the image, and then re-synthesizing the sound from the altered spectrogram. Many were inspired by Xenakis' UPIC system which was designed around the second half of this workflow: you'd draw the spectrogram with a pen and then synthesize it.

https://en.wikipedia.org/wiki/UPIC

Edit: my favorite of all these systems was Chris Penrose's HyperUPIC which provided a lot of freedom in configuring how the analysis and synthesis steps worked.

[ Parent | on HN ]