You're right, I'm not arguing against the article so much as trying to continue its thought. I agree that generative models are useful. Among people who use them, I suspect that the next major predictor of success would be adaptability. Confirmation bias is the opposing force: once we form a model, we are biased towards it, and tend to overlook contradictory evidence.
One tactic that increases adaptability is not forming a model until you actually have to; this avoids confirmation bias.
You're also right about the nature of the test: as an experiment, it does show that presence of generative models predicts success. I'm interested in education, so I'm voicing the reason why this wouldn't be a practical test of qualification.
Re:Generative models and programming talent
by shevertalov on Tuesday July 22, 2008 @04:09PM
(#253591)
The bias point is well made. I don't remember the quote exactly, but i think it's something like "...when all you know are hammers, everything looks like a nail...". Though I think his point is that you have to at least be able to form models about the problems you tackle and stick to those models. I feel like you need some commitment or vision to be able to move on. Development tends to have two phases, exploration and exploitation. Balancing the two is a real problem. If you exploit too soon you may miss a much better solution, but if you do too much exploring you may just be wasting time. Balancing the two contributes to writing good code it seems.
I think you are also using a different definition of "optimal strategy" than the author. The test was not measuring test-taking strategy - it may not even have been scored in a student-visible way. In the absence of a score-optimizing approach the natural inclination of the student's analytic style should come to the forefront. Additionally, the test had predictive power regardless of a root cause. Students who used a consistent model in selecting answers went on to succeed in the programming course, and the others did not. The underlying cause may actually be different but the concrete result stands.