Drawbacks? The difference could also be positive. Think of toki pona, an artificial language with only 123 words and their compounds.
http://en.tokipona.org/wiki/What_is_Toki_Pona%3F
When you see a bug, you a bug, whether it's a cockroach or a spider. Thinking becomes clearer and simpler (or so they say).
A single word can hold a large amount of meaning, but to the brain that single word uses the same "storage" and any other word.
So by having many complex words a person can think more "efficiently".
This is why many fields have jargon - the jargon helps the thinking process, by essentially acting like file compression - you compress an entire concept into a single word, and you can now think about some big complicated thing by using a single word.
This is also why language have so MANY words. English has something like half a million of them. Yet it would be totally possible to replace the vast majority of them with simpler, but more, words.
But doing so has drawbacks to thinking ability.