I am not usually on the techno-optimist side of things but I have a hard time believing, after observing the progress of SOTA in the past decade on the big problems explored by ML research, that we are more than 5-10 years out from exceptional generative samplers based on tiny snippets of audio.
You basically need a breathe controller to make a realistic playable sax sound otherwise you can't control the variables in real time that are needed.
Personally, I want to hear synth sounds that are not even trying to be reproductions of acoustic instruments. If I want to hear a sax sound I want to hear someone who is really good on a real sax and who knows how to actually play a sax.
Pianos are actually really easy, input-wise. You have attack speed, when you lift the keys and pedals, and that's it. But simulating the audio computations done by a physical instrument is just really difficult.
There are some physically-based (spherical cow physical) synths that sound more alive than the usual sample-based stuff, but their timbre is not right.
If I had more time, this is what I'd be working on right now. There's a market for a great piano synthesizer.