Coming a bit late to the discussion, but out-of-vocab words and small amounts of labelled data is the exact kind of problem that word embedding vectors are meant to solve, and they do so exceedingly well. Of course, you need a good, large source of (unlabelled) data, preferably approaching the gold standard of "everything that has ever been written about your problem domain" to train the embeddings, and that takes some effort, but then the out-of-vocab words must be exceedingly rare (i.e. only if it's a genuinely new word that has never ever been mentioned before, or a very very rare term that you've discarded), and even that can be solved well with character-based embeddings.