Re:MusicLM: Generating music from text
by not-chatgpt on Thursday January 26, 2023 @11:35PM
(#34542527)
Their current demos seem worse than riffusion results I get on average. Music gen is hard because music is inherently a composed of many many different instruments, each with unique sound and function. Simply training end-to-end will almost always end up badly.
[ Parent
| on HN ]
Re:MusicLM: Generating music from text
by robitsT6 on Thursday January 26, 2023 @11:40PM
(#34542546)
I'm sorry, I like riffusion as much as the next guy, but in what world is any riffusion example better than the literal first example on this demo page?
Re:MusicLM: Generating music from text
by not-chatgpt on Friday January 27, 2023 @12:19AM
(#34542779)
Might be personal preference, but I don't think those examples are good at all. The only thing I am impressed with is the story mode and conditioning.
I typically use riffusion to generate swing/electropop music, could be I'm too biased.
> Music gen is hard because music is inherently a composed of many many different instruments, each with unique sound and function. Simply training end-to-end will almost always end up badly.
Yes! That's what I figured when I started my project to create an AI assistant for melodies (http://melodies.ai/). I'm quite sure that splitting it up is the way to go, and that the first AI hit song will not be created as a full song but by combining instruments/vocals.
FYI, they don't actually train end-to-end, they have three separate models that are each trained via self-supervised learning independently (I think, I haven't read the paper very carefully).