News for hackers. Stuff that matters.

A comment on Riffusion – Stable Diffusion fine-tuned to generate Music. Parent comment

Re:Riffusion – Stable Diffusion fine-tuned to generate Music by haykmartiros on Thursday December 15, 2022 @11:06AM (#34001592)
Yes from https://huggingface.co/runwayml/stable-diffusion-v1-5. Our checkpoint works with automatic1111, and if you'd like to make an extension to decode to audio, it should be pretty straightforward: https://github.com/hmartiro/riffusion-inference/blob/main/ri...
[ Parent | on HN ]
Re:Riffusion – Stable Diffusion fine-tuned to generate Music by Metus on Thursday December 15, 2022 @11:17AM (#34001740)
Can you run this on any hardware already capable of running SD 1.5? I am downloading the model right now, might play with this later.

Guessing at the speed with which AI is developing these days someone is going to have the exension up in two hours at most.

[ Parent | on HN ]
Re:Riffusion – Stable Diffusion fine-tuned to generate Music by ronsor on Thursday December 15, 2022 @11:22AM (#34001803)
I bet the AUTOMATIC1111 web UI music plugin drops within 48 hours.
[ Parent | on HN ]
Re:Riffusion – Stable Diffusion fine-tuned to generate Music by enlyth on Thursday December 15, 2022 @09:40PM (#34009035)
I have made a basic version here:

https://github.com/enlyth/sd-webui-riffusion

[ Parent | on HN ]
Re:Riffusion – Stable Diffusion fine-tuned to generate Music by haykmartiros on Thursday December 15, 2022 @11:24AM (#34001830)
Yes! Although to have real time playback with our defaults you need to be able to generate 40 steps at 512x512 in under 5 seconds.
[ Parent | on HN ]
Re:Riffusion – Stable Diffusion fine-tuned to generate Music by Metus on Thursday December 15, 2022 @11:32AM (#34001942)
Good to know. I was just so close with just under 7s using 40 steps and Euler a as sampler.
[ Parent | on HN ]