For my final point, you can model this as a hidden markov process. You're observing notes, but not the user moving his mental "cursor". Classical filtering algorithms should do well here.
Ah, one more missing feature, good scores have fingering, but it gets lost in the midi. I'm pretty sure fingering can be infered though and this would be pretty powerful. Amusingly, it can also be seen as a filtering problem:
- At any given point, a hand can have a finite number of fingerings, and that number isn't that large. Give each of them a difficulty.
- Now, make that difficulty conditional on the previous hand position.
- Run Viterbi's algorithm on the score to minimize the difficulty.
The difficulty can be obtained from an expert or through some physiological model, but they could also be inferred from an existing body of music sheets.
Importing alternative formats (xml for instance) is definitely an option worth looking into, but because the bulk of the material that is out there that people can freely access is in midi format we chose that one first.
Musescore already has such a collection of sheet music https://musescore.com/hub/piano so it'd also be possible to allow users to import from there, going from sheet music to midi instead of trying to reverse the process.
As using MIDI will probably never be perfect, have you thought about using the score in a suitable format, like LilyPond, directly?
A midi recording contains pretty precise velocity info for each note (0-127) that is either not present at all in a score, or is present only in the form of a limited number of dynamic markings[1], which govern the flow of dynamics imprecisely, and almost not at all on a per-note basis.
With tempo it is similar to dynamics: midi will precisely determine when notes turn on and off, down to the millisecond, in the process resulting in the midi performance having a precise tempo. In contrast, musical scores have limited means to indicate proper tempo[2], necessitating (as with dynamic markings in a musical score) much interpretation by the musician.
All of this is not to say that a midi performance is somehow "better" than a score. There are good midi recordings and bad ones. A good recording will generally be made by an accomplished musician who in making the recording records fairly precisely a particular (and good) interpretation of the score. Likewise, a bad recording would generally be the result of a poor interpretation of a score. But whatever the case, the midi recording is far more precise than the score itself.
[1] https://www.aboutmusictheory.com/music-dynamics.html
[2] https://en.wikipedia.org/wiki/Tempo#Basic_tempo_markings
A midi file is designed to contain different kinds of information than a score, so it can absolutely contain less information (of a certain kind) than a score.
Imagine a group of notes that are attacked very close together in time in a midi file - is it meant to be a chord played without arpeggiation? A chord arpeggiated as fast as possible? A chord arpeggiated to a specific tempo?
When a composer writes (say) a chord with an arpeggiation sign next to it, it specifies that the notes are to be played in a certain order, and quickly. There are multiple possible performances that comply with this "specification". MIDI cannot encode this information (the information being the set of acceptable performances).
Midi is 'lossy' in the sense that it flattens a bunch of knowledge about how a piece is to be rendered into note-on/note-off pairs. You then have to do all kinds of tricks to even begin to approach what the score looked originally. Lots of enharmonically (identically sounding) combinations may have looked completely different on the original score, and when you start adding the various chord combinations it gets even more complicated. Even so, I'm strongly convinced it can be solved and we have made considerable progress on this already.
The 'wiggle room' aspect of it has been addressed to some extent. For instance, we build up a map of all the notes that are sounding at a given time, and will ignore notes that are technically 'wrong' but that are still sounding. This allows for much smoother restarts after mistakes.
Your final point is a bit tricky, but thank you for the hint, I will definitely add it to the list of things to look at to see if we can find a way to make that work.