Re:Diffusion Is Spectral Autoregression
by nyanpasu64 on Tuesday September 03, 2024 @05:49PM
(#41439636)
(not the author) There's techniques for consistent-phase audio synthesis like phase vocoders, but they are beyond my current knowledge.
Re:Diffusion Is Spectral Autoregression
by benanne on Tuesday September 03, 2024 @08:45PM
(#41440761)
I actually wrote down some thoughts about audio phase in a previous blog post: https://sander.ai/2020/03/24/audio-generation.html#motivatio...
I have an example audio clip in there where the phase information has been replaced with random noise, so you can perceive the effect. It certainly does matter perceptually, but it is tricky to model, and small "vocoder" models do a decent job of filling it in post-hoc.
So when generating audio I think the next chunk needs to be continuous in phase to the last chunk, where in images a small discontinuity in phase would just result in a noisy patch in the image. That's why I think it should be somewhat like video models, where sudden, small phase changes from one frame to the next give that "AI graininess" that is so common in the current models