Yes, I get what you want. This approach is used in many modern synthesizers, such as the Korg Kronos. It's just that approach proved to be too complicated to get working well for speech to text. I was a little surprised myself. Google for vocal tract modeling and you'll see it pretty much died out a decade or so ago.