In which Pat Metheny tries to replace his whole band with guitar-triggered control of real, non-guitar instruments (keyboards, drums, etc) ... "Orchestrion" ... https://youtu.be/KsYEOUKS4Yk

Yeah, it's honestly not that good. I think Metheny uses an Axon. Even with those you need to be very careful with your phrasing, you can't be too fast, you lose a lot of dynamic range and a lot of expressivity, sometimes notes just die, the latency is high...

Here's what Metheny said himself: "But the guitar‑to‑MIDI part has always been a problem. It's a question of physics. On input, I sort of have to rush. But I know how to rush. I play ahead."

It's fun for lots of things, and you can make lots of cool music, but it's still very limited to certain styles, dynamic ranges, phrasing, tempo, speeds...

polyphonic note detection is largely solved at this point.

But "solved" here means "when not doing the analysis in real time". The realtime solutions are not as good. NN's are not typically great at realtime either, so this may not help very much with this particular goal.

The video in TFA is in real time, so I don't believe your assumption is correct.

I don't think TFA involves polyphonic note detection, just effects processing of the whole audio signal.

I never claimed or implied anywhere that it does, and I know for a fact TFA doesn't involve note detection.

I'm just disputing the GP general assertion about NNs that "NN's are not typically great at realtime either", which is quickly disproven by the TFA which uses NN for realtime audio.