News for hackers. Stuff that matters.

A comment on Riffusion – Stable Diffusion fine-tuned to generate Music. Parent comment

Re:Riffusion – Stable Diffusion fine-tuned to generate Music by echelon on Thursday December 15, 2022 @12:24PM (#34002768)
Griffin-Lim is slow and is almost certainly not being used.

A neural vocoder such as Hifi-Gan [1] can convert spectra to audio - not just for voices. Spectral inversion works well for any audio domain signal. It's faster and produces much higher quality results.

(Source: author of FakeYou.com, and we're about to launch a music product.)

[1] https://github.com/jik876/hifi-gan

[ Parent | on HN ]
Re:Riffusion – Stable Diffusion fine-tuned to generate Music by antognini on Thursday December 15, 2022 @12:31PM (#34002851)
If you check their about page they do say they're using Griffin-Lim.

It's definitely a useful approach as an early stage in a project since Griffin-Lim is so easy to implement. But I agree that these days there are other techniques that are as fast or faster and produce higher quality audio. They're just a lot more complicated to run than Griffin-Lim.

[ Parent | on HN ]
Re:Riffusion – Stable Diffusion fine-tuned to generate Music by seth_ on Thursday December 15, 2022 @02:50PM (#34004688)
Author here: Indeed we are using Griffin-Lim. Would be exciting to swap it out with something faster and better though. In the real-time app we are running the conversion from spectrogram to audio on the GPU as well because it is a nontrivial part of the time it takes to generate a new audio clip. Any speed up there is helpful.
[ Parent | on HN ]