News for hackers. Stuff that matters.

A comment on Riffusion – Stable Diffusion fine-tuned to generate Music. Parent comment

Re:Riffusion – Stable Diffusion fine-tuned to generate Music by woah on Thursday December 15, 2022 @11:42AM (#34002147)
You're probably talking about the artifacts of converting a low resolution spectrogram to audio.
[ Parent | on HN ]
Re:Riffusion – Stable Diffusion fine-tuned to generate Music by wdfx on Thursday December 15, 2022 @12:06PM (#34002508)
Can the spectrogram image be AI upscaled before transforming back to the time domain?
[ Parent | on HN ]
Re:Riffusion – Stable Diffusion fine-tuned to generate Music by malka on Thursday December 15, 2022 @12:24PM (#34002772)
Yes it exists: https://ccrma.stanford.edu/~juhan/super_spec.html

But the issue is not that the spectrogram is low quality.

The issue is that the spectrogram only contains the amplitude information. You also need phase information for generating audio from the spectogram

[ Parent | on HN ]
Re:Riffusion – Stable Diffusion fine-tuned to generate Music by mcbuilder on Thursday December 15, 2022 @12:37PM (#34002921)
Interesting, can't you quantize and snap to a phase that makes sense to create the most musical resonance?
[ Parent | on HN ]
Re:Riffusion – Stable Diffusion fine-tuned to generate Music by waltbosz on Thursday December 15, 2022 @01:29PM (#34003599)
What happens if you run one of the spectrogram pictures through an upscaler for images like ESRGAN ?
[ Parent | on HN ]