Polyphonic sound decomposition is pretty much an open problem. I have an 'interesting' idea on how to solve it. I'm sure I'll be accused of cheating if I get it to work ;)

Oh, that's true (Melodyne made a plug in that will do it, AFAIK). Can't you do something else, like optically or mechanically read the keys? Do you even need to read them? I guess you want to tell people if they made a mistake, hmm...

Something you could do would be IR LEDs and an IR camera, where the LEDs are only reflected on the shiny keys when they're pressed due to the angle.

Hehe, 'why don't you' :)

Yes, there are many use cases that I have in mind where this is a must. In fact, if I can't crack it then it's a useless project (other than that I will learn from it).

There are all kinds of devices that could give you midi inputs (such as the 'Piano Bar', sadly discontinued), but that's the easy way out. After all, you could also simply use a MIDI enabled keyboard. But I want it to work with a real piano, as well as possibly other instruments.

The best software for this at the moment is Anthemscore.

Hmm, very true. Do you want it to not require modifications to the piano? I guess that would mean that you pretty much need to do sound decomposition...

Yes, zero hardware changes or mods to the piano. I think that would be a dealbreaker for many people. It also means that the barrier to entry would be much lower.

Hmm, yes, if you can get it to work with just sound, that would be trivial to set up. If not, I think the light idea would still be valuable, as technically you wouldn't be changing the piano (just adding a camera and light above it).

I've actually toyed around with that idea, one interesting application would be to convert Rousseau style youtube videos back into midi.

I'm quite sure it is possible but it would still require a lot of set-up. The microphone is such a temptingly simple target that it makes sense to pursue that one first.

That makes sense (although the microphone is only a simple target if you ignore the unknown unknown of deconstructing the sound :P). I'm very much looking forward to your progress!