I've come across AI-generated music that outputs something like MIDI and controls synthesizers. Its audio quality was crystal-clear, but the music was boring. That's not to say the approach is a dead-end, of course -- and indeed, as a musician, the idea of that kind of output is exciting. But getting good data to train something that outputs separate MIDI-ish voices seems much harder than getting raw audio signals.
It’s easy to forget this is all pretty new stuff and it still costs a lot to make the base models. But the techniques are (more or less) well documented and implementable with open source tools.
Note sequence generation from statistical models has a long history, at least as long if not longer than text generation.
Have a look at section 2.1 of this survey paper [0] that cites a paper from 1957 as the first work that applies Markov models to music generation.
And, of course, plenty of follow-up work 6 decades later on GANs, LSTMs, and transformers.
[0]: https://www.researchgate.net/publication/345915209_A_Compreh...
Progress only moves as fast as expectations, and expectations move with technology. Music is not special in this respect. So you could say at any given time in the past that some people "see meaningfully high quality" and others are disappointed. You see exactly both these sides of the spectrum even now with text-to-image and text-to-audio technology.
But that makes me wonder to what extent labeling can be programmed -- extracting chord changes, dynamics changes, tempo, gross timbral characteristics, etc.
By Fred Brooks no less…
I really like this idea. Creating new tools for artists to use to create rather than whatever we're accepting as use now. The use of current full image creation is boring to me in the same way the choice of invisibility as a super power is. The invisibility is ultimately going to slide into pervy tendencies, just like deep fakes will slide in the same way or some other inappropriate use.
This is what we do with AI images: you can fix them in Photoshop, etc. You cannot do this for raw audio due to how music is produced.
When that paper was written it took several weeks and many millions of dollars of equipment to render primitive, mono-color 3d graphics. Desktop computers had 512 kilobytes of RAM and the highest-end desktops 32 MB of hard drive storage space. Computer screens had two colors: black and green. Audio cards capable of making beeps and clicks were the cutting-edge. WIFI was still a decade away.
"Intention"
The question becomes: what has impeded the creation of a MIDI file that can be confused with an actual concert from Arturo Benedetti Michelangeli.
The current version of MIDI is capable of replicating any of his performances, even down to the randomness.
Note that if you want to replicate the audio quality of his performances, you will need a high-quality MIDI instrument; the ones that ship with Windows will not suffice. These MIDI instruments can range from a few dollars to thousands of dollars. (See, e.g., Native Instruments)
In such case, we have a theoretical suggestion that «nothing is preventing this», but not an actual proof based on a "Turing test"-like scenario which would have specialists fooled, to corroborate that the new MIDI 2 would suffice.
As it is clear that you intend to keep shifting the goalposts to make a point that can't be made, I will withdraw from further participation in this thread.
Midi has to fire all the notes one at a time. You cannot represent even everything represented by a score through MIDI because it's serial and sends one message at a time. Stretch that out to 16 sharp clicky attacks and you'll notice that MIDI cannot fire them all at once, it'll blur.
However, if you want to control attack, reverb, tone, volume, i.e., all the stuff that makes it seem like a real instrument and not a computer-generated tone, then you would need to dedicate some of the 16 channels on each port to each of those different controls. To handle 16 xylophones, you would need a MIDI interface with enough ports to handle the note data plus however many control channels you want to use. (Note that from the interface's perspective, the data incoming from 1 port is just a single channel, so an "8-channel" MIDI interface is actually handling 128 channels of MIDI data.)
All these messages have to take turns. It's not a tracker, or a modular synth, where you can parallel click rise times electronically: it's not DINsync, where a chain of voltage pulses synchronize individual sequencers and aren't themselves notes.
1000 messages a second (at three bytes for each note-on) seems like a lot but it really isn't. With your 16 instruments (any drum, the xylophones, whatever) you can fire about 62 notes on all instruments per second. That seems like a lot too, but it's a hard limit, and it means your sixteen instruments have to 'blur' across 16 milliseconds to all fire off a note.
That means every time you fire all the instruments as one click, instead the attacks make up a 960hz tone. That's NOT one attack. Humans on percussion instruments can do better than that. It's the equivalent of 5.5 feet of space between speakers playing back: if you're time-aligning a tweeter and a midrange to produce a unified click, and you misalign one of the drivers by putting it five and a half feet back from where it would be, you'll notice the misalignment. Midi's timing issues are also noticeable.
Consider that Hollywood composers and music producers are able to use MIDI without issue for live-previews of orchestral compositions and live recordings and performances. The problem you are raising simply doesn't exist in the real world, accept that there are things about MIDI you don't understand and move on.
Still ain't quite DINsync or modular, but there ya go :)
It’s then for the remaster / musician / actual interpretation / post production to make the score to something less of an event model.
In practice, though, transformations from audio to MIDI discard an enormous amount of important information, with the possible exception of transcriptions of performances on piano (where volume, frequency, duration and a good physical model of a piano are enough to reconstruct everything important about the signal) and similar instruments.
Contemporary MIDI is ready to describe almost any human performance in an almost absurd level of detail. Whether anything can do something useful with the description is a different story.
While this does not seem to be the trend I hope more gen ai in the audio and visual realms start to produce more structured / symbolic output. For example, if I were Adobe I would be training models, not to output full images, but either layers or brush strokes and tool pallet usage. Same for organizations that have all the component tracks of music to work with.