I’m super curious, if you don’t mind my prodding. I looked into doing this for my own version of a JavaScript midi teacher a while back. Any chance you’re doing it via the neural nets sequence to sequence translation algorithms that are super popular in NLP right now? I think that’s a practical way to do it, except for needing a training corpus. I was going to reverse engineer synthesia’s data to try it out, for what’s worth...

We do not think that it will need a neural net in order to be able to do this, but we need to prove that in order to make it stick and we're not quite ready to do that. Keep in mind that this is in every sense of the word a side project, we only work on this when we do not have technical due diligence work for customers and that still continues quite strongly. But on 'off' weeks we work on this and the project progresses quite rapidly when we do.

I think it could be done with a constraint solver, with a lot fewer cycles.

This is the route we are looking at, with some twists.