News for hackers. Stuff that matters.

A comment on Riffusion – Stable Diffusion fine-tuned to generate Music. Parent comment

Re:Riffusion – Stable Diffusion fine-tuned to generate Music by hahajk on Thursday December 15, 2022 @05:53PM (#34006957)
I think this type of generative stuff opens up entirely new possibilities. For the longest time I've wanted to host a rowing or treadmill competition, where contestants submit a music track. The tracks are mashed up with weighting based on who is in the lead and by how much.

I don't know of existing tech that can generate actual good mashups in realtime given arbitrary mp3s, but this has promise!

[ Parent | on HN ]
Re:Riffusion – Stable Diffusion fine-tuned to generate Music by refulgentis on Thursday December 15, 2022 @06:07PM (#34007100)
no, because is a function ("AI") that generates an image of a spectogram given text.

neither a set of MP3 nor a set of spectrograms from MP3s supplies the function arguments

or a connection to a path that uses that function

[ Parent | on HN ]
Re:Riffusion – Stable Diffusion fine-tuned to generate Music by astrange on Thursday December 15, 2022 @08:28PM (#34008348)
It says all StableDiffusion capabilities work, so you can prompt it with an image (either "img2img" or "textual inversion"). Their UI just doesn't expose it.
[ Parent | on HN ]
Re:Riffusion – Stable Diffusion fine-tuned to generate Music by marktolson on Thursday December 15, 2022 @06:49PM (#34007509)
It's not too hard these days with open source BPM detection and stem separation libraries: https://github.com/deezer/spleeter
[ Parent | on HN ]