dsanderson

Sorry for the delay, I didn't realize you had replied (is there some global notification UI element I should look for?) I don't know if you'll see this, but I'll reply anyway.

I'm not from California, nor have I studied their tests, nor did I claim that standardized tests themselves are flawed, so your first demand is misleading, at best. I will grant that, having done further research, my first examples of teaching to the test were flawed; I'll try to summarize it (and my position) below.

Teaching for the test is explained pretty well here: http://www.carnegiefoundation.org/perspectives/teaching-test and on wikipedia [ http://en.wikipedia.org/wiki/Teaching_to_the_test ]. Note that this definition of teaching to the test isn't necessarily bad, only when applied wrong. Punishing/rewarding schools based on student performance on a small set of tests creates an incentive to have student "learn" by mercilessly practicing on those tests, to the exclusion of other approaches. As per the sources on wikipedia, this doesn't impart a "general understanding," nor does it necessarily raise test scores. Other people here have linked stories where teachers and schools actually helped their students cheat on standardized tests. Standardized tests themselves aren't the problem; we've had them for decades, and they can provide valuable insights in some cases. The problem is when we focus on standardized tests to the exclusion of all else, putting a perverse pressure on schools. I believe you previously supported evaluating by standardized tests because they're the only quantitative measure we have; maybe this means we can't reliably quantitatively evaluate the impact of a single teacher on a small set of heterogeneous subjects. Demonstrate we can, then I'll reconsider NCLB.

edit to add another thought: I'd liken this to the problem of overfitting in machine learning. You have a way of measuring the performance of your system, and try to improve the system via that feedback. It's easy to overfit it, making it work beautifully on the provided data, but lack generality. The problem isn't in how we measure "fitness," necessarily, it is in how we update the system based on that result. That is, standardized tests may be a fine measure among students with that "generalized understanding"; I've not been convinced we are using those results intelligently.

Standardized tests claim to measure those quantities. To be fair, my school system was quite good, so I don't think I suffered too greatly from teaching to the test, but I did read quite a few of the study/test guides put out by Kaplan and their ilk. Much of it was tricks based on the structure of the tests; the "types" of question that will be asked and type-specific solution strategies. These tests are not about basic understanding and skill at adding, reading comprehension, etc., but rather a well-defined subset of those skills. The assumption is that skill in the subset is a good proxy for fundamental skills in the student. If teachers are evaluated solely by these partial measures, it is in their best interest (and, in the eyes of the administrators, produces the best results) if they optimize for these partial measures; the argument from standardized testing opponents is that teaching for these partial measures has not been shown to produce general understanding.

In that sense, NCLB created a race to the bottom; schools that performed below-average got less funding, so if any schools successfully "taught to the test", they all had too or face budget cuts (that's my understanding, IANAT).

Is there evidence that these partial measures reveal and/or necessarily lead to a more general understanding?

Yeah, I modified an old CVS disposable digital camera to do this once. Those filters only block near IR (as mentioned in the article), basically light just outside our vision spectrum. You can use infrared LED's and a camera thus modified to make a cheap night vision camera (This is, I believe, how the night mode on old sony cameras would work).

Measuring heat, however, requires far IR, which won't be picked up by removing the filter.