That would be really interesting to me as well. Do you know of any other project or startup working on that?

Similar to the OpenCV library for computer vision, I wish there were an "OpenVC" for voice control.

The pocketsphinx-specific code is actually quite simple:

https://bitbucket.org/davidn/dom/src/default/listen/listen.c

You can see the keyword and jsgf files in that directory, for reference. The pronunciation dictionary is generated from one of the standard dictionaries, selecting just the works present in the grammar.

Note: If you look in the whole directory, there's a mix of decent code and ugly hacks in there, and I didn't make any attempt at making it customizable. It's just for me. Btw, it all runs on a raspberry pi.

Btw. their probabilistic parser (http://goo.gl/rRdRx4) might be useful for your project: "wake me up the first Friday of February 2014 at 7am"

Great to hear that RaspberryPi (and Python) is fast enough for it.

Basically, you can define vocabulary sets to check against, and you provide a callback to know when a word has been detected. Depending on the complexity of the vocabularies, you might need to spring for the Rejecto plugin to improve the results.