The amazing thing is that the current diffusion models are so good that the spectograms are actually reasonable enough despite the small room for error.
I think this will be particularly useful for musical compositions in movies and film, where the producer can "instruct" the AI about what to play, when, and how to transition so that the music matches the scene progression.
If you're interested, the idea of applying Image processing techniques to Spectrograms of audio is explored in brief in the first lesson of one of the most recommended AI courses on HN: Practical Deep Learning for Coders https://youtu.be/8SF_h3xF3cE?t=1632
Also, the music doesn't sound "Lofi" because it's generated by algorithms. A lot of hard work and software goes into taking a clean, pitch-perfect digital signal and making it sound like something playing on a record player from the 70s.
I just ran a quick Google Scholar search, and the first result is https://ieeexplore.ieee.org/abstract/document/5672395
This is from 2010. I didn't go looking, but it wouldn't surprise me if the idea is older than that.
https://en.wikipedia.org/wiki/UPIC
Edit: my favorite of all these systems was Chris Penrose's HyperUPIC which provided a lot of freedom in configuring how the analysis and synthesis steps worked.
Sure, AI can do lots of things well. But would you rather live in a world where humans get to do things they love (and are able to afford a comfortable life while doing so) or a world where machines do the things humans love and humans are relegated to the remaining tasks that machines happened to be poorly suited for?
I think that kind of attitude is defeatist - it's implying that humans will be stopped from making music if AI learns how to do it too. I don't think that will happen. Humans will continue making music, as they always have. When Kraftwerk started using computers to make music back in the 70s, people were also scared of what that will do to musicians. To be fair, live music has died out a bit (in a sense that there aren't that many god-on-earth-level rockstars), but it's still out there, people are performing, and others who want to listen can go and listen.
Maybe consumers will start consuming more and more AI music, instead of human music [0], but the worst thing that can happen is that music will no longer be a profitable activity. But then again, today's music industry already has some elements of the automation - washed-out rhythms, sexual thematics over and over again, re-hashing same old songs in different packages... So nothing's gonna change in the grand scheme of things.
For me, the worst that could happen is that people spend so much time listening to AI generated music, that human musicians can no longer find audiences to connect to. It's not just about economics (though that's also huge). It's the psychological cost of all of us spending greater and greater fractions of our lives connected to machines and not other people.
Black metal community, for example, has always rejected all forms of "automation" and considers it not kvlt - rawness is a sought-after quality, defined as having people performing as close to the recording equipment as possible.
There's also a rapper named Bones (Elmo O'Connor) who's never signed a contract with a label, does only music he wants to do, releases a couple albums every year. There's something about his approach that makes his music sound very organic and honest. I listen to him more than I listen to any mass produced rapper.
So in conclusion, music was always about people. Unless AI reaches AGI level, I don't think it will ever impact music enough to kill all audience.
Advancing AI capabilities in no way detracts from this. You talk about humans being "relegated to the remaining tasks" - but that's a consequence of our socioeconomic system, not of our technology.
Those two are profoundly intertwined. Our tech affects our socioeconomic systems and vice versa.
I don't see this as a threat to human ingenuity in the slightest.
I can't wait to hear some serious AI music-making a few years from now.