It's like a ghost playing. Absolutely crazy to hear his touch but with modern fidelity.
For instance, how long you are touching the string - while you're touching the string, there is a sound - but after you let off, you get the "reverb" - and there is different reverb for how you hit the key, if you bounce, or if you stay for a split second longer for staccato, I don't feel like these subtleties translate to MIDI.
It is certainly easier than violin, that I will grant.
edit: IMO the best way to do what you are describing is get a really good pianist to sit down and do the work. I don't think that (current?) machine learning can really "understand" the nuance of phrasing esp that would be coming from older recordings.
But wouldn't they be reproduced when replaying the MIDI data physically on a piano?
Ultimately isn't how you hit the key and bounce/stay still just initial velocity and then timing of letting go? Perhaps the velocity of letting go would have to be added as well, but I'm not actually sure if that's really acoustically meaningful.
I guess I don't see why all the reverb and ultimate sound complexity wouldn't be recreated in playback? Of course, this requires actual physical playback on a similar enough model of piano, or else a synthesizer that is sufficiently accurate.
The third (middle) pedal in pianos is nonstandard -- i.e. used for different effects on different pianos, whether sostenuto or bass damper or practice mute.
In actual performance the only time it's ever really used (and rarely at that) is as sostenuto, since that's what it does on grand pianos like Steinways, but its effect is indistinguishable from simply holding notes for longer durations, so MIDI can simply represent its effect that way. (Unlike the sustain pedal which increases resonances in a big way and needs to be represented independently, or soft pedal which changes timbre as well as volume.)
But it looks like already people have attempted the transcription strategy you describe https://www.google.com/search?hl=en&q=machine%20learning%20n...
I love whenever it's possible to upsample/restore media due to known constraints in the original -- in this case, how screens work.
Something analagous I've been waiting for is regenerating old scratchy piano recordings. Piano is unexpectedly simple compared to other instruments -- the only inputs are really note down + speed, note up, sustain pedal pressure, and (less frequently) soft pedal pressure.
Seems like you should be able to turn any solo piano recording, no matter how degraded, into a relatively lossless MIDI representation, then re-record that replayed physically (via motors, which exist already) on a modern piano, or even just synthesized, trying to be as true to the original piano's characteristics as possible. Losing literally none of the artistry.
It seems like this should be "easy" for piano in a way that it isn't, for example, with violin which has so many more complicated characteristics of pitch, timbre, bowing, vibrato, etc.