Re:Alien Dreams: An Emerging Art Scene
by songeater on Friday July 02, 2021 @12:15PM
(#27713427)
agreed. full waveform "music" (unlike say MIDI) is just many more variables than a "picture" or even "video". Also you have to "stitch together" a lot of these samples to get anything that resembles a "new" song. More akin to mining... but still its kind of crazy what can be done with it.
[ Parent
| on HN ]
>> Why doesn't the architecture use a model to generate sheet music / MIDI, then layer another model on top to create instrumentals?
there are various models that do this [0]... just doesn't have the same power to generate waveform music that jukebox does.
Jukebox effectively is a 2 step model a) a 3 layer VQ-VAE compresses the music (so think of this as the MIDI equivalent) and b) a transfomer then learns/generates sequences.
The compression is the expensive part of the model.
[0] https://towardsdatascience.com/how-to-generate-music-using-a...
I'm not familiar with Jukebox, so maybe I'm wrong. It seems like you'd want a fast architecture to use a model to generate sheet music / MIDI, then layer another model on top to create instrumentals?