This idea seems to strike many different people at different times. One my favorites is the ANS synthesizer [0], made with optical tone wheels. There's an android app simulating it [1] (my only relation with it is "user"). The synth presented in this github page really goes a step beyond.

[0] https://en.m.wikipedia.org /wiki/ANS_synthesizer

[1] https://play.google.com/store/apps/details?id=nightradio.vir...