That feels to me like stacking challenges because image interpretation is a challenge on top of concept generation as a challenge.
That said, there is the problem of believing your initial concepts. And, like people, if you start with a bunch of bogus concepts its going to be hard to break free of that and establish concepts more liberally. I think about it as the question of not only establishing the concepts but establishing the validity of the concepts that have been established. In a very sparse concept space your "best match" can be really far off from what someone with a more filled out concept space would consider valid.
We have hundreds of thousands of hours of programming right from the Kindergarten level.