The subjective opinions of a teacher are not a metric, they are merely a noisy data set.

They are also nearly worthless scientifically - they are incomparable (how do you compare the opinion of teacher A to that of teacher B) and non-reproducible (how can one repeat an observation).

All true, of course. But numerically accurate data on a proxy quantity that you don't care about are not necessarily inferior to numerically inaccurate data on the quantity you care about. Both pose challenges to interpretation. It's easy to be seduced by clear numbers and worry less about what the numbers mean. We can argue about which fields count as "science", but many fields try to learn from non-reproducible data.

(And I say this having spent my career in the physical sciences. There's too much noise induced by people who present statistically solid analyses of numbers whose connection to the core questions is much more tenuous...)

Standardized tests typically measure how good a student is at adding, subtracting, and reading a paragraph and using the information it contains.

Why do you feel we don't care about that? What information should we care about?

Standardized tests claim to measure those quantities. To be fair, my school system was quite good, so I don't think I suffered too greatly from teaching to the test, but I did read quite a few of the study/test guides put out by Kaplan and their ilk. Much of it was tricks based on the structure of the tests; the "types" of question that will be asked and type-specific solution strategies. These tests are not about basic understanding and skill at adding, reading comprehension, etc., but rather a well-defined subset of those skills. The assumption is that skill in the subset is a good proxy for fundamental skills in the student. If teachers are evaluated solely by these partial measures, it is in their best interest (and, in the eyes of the administrators, produces the best results) if they optimize for these partial measures; the argument from standardized testing opponents is that teaching for these partial measures has not been shown to produce general understanding.

In that sense, NCLB created a race to the bottom; schools that performed below-average got less funding, so if any schools successfully "taught to the test", they all had too or face budget cuts (that's my understanding, IANAT).

Is there evidence that these partial measures reveal and/or necessarily lead to a more general understanding?

Here is an actual CA standardized test:

http://www.cde.ca.gov/ta/tg/sr/documents/cstrtqmath7.pdf

Please explain why you feel it doesn't adequately measure the ability of students to "read, write, and compare rational numbers in scientific notation (positive and negative powers of 10) with approximate numbers using scientific notation" (as well as the other topics listed).

Or, similarly, can you explain specifically what "teaching to the test" means and why it is suboptimal for learning the aforementoined topics?

Sorry for the delay, I didn't realize you had replied (is there some global notification UI element I should look for?) I don't know if you'll see this, but I'll reply anyway.

I'm not from California, nor have I studied their tests, nor did I claim that standardized tests themselves are flawed, so your first demand is misleading, at best. I will grant that, having done further research, my first examples of teaching to the test were flawed; I'll try to summarize it (and my position) below.

Teaching for the test is explained pretty well here: http://www.carnegiefoundation.org/perspectives/teaching-test and on wikipedia [ http://en.wikipedia.org/wiki/Teaching_to_the_test ]. Note that this definition of teaching to the test isn't necessarily bad, only when applied wrong. Punishing/rewarding schools based on student performance on a small set of tests creates an incentive to have student "learn" by mercilessly practicing on those tests, to the exclusion of other approaches. As per the sources on wikipedia, this doesn't impart a "general understanding," nor does it necessarily raise test scores. Other people here have linked stories where teachers and schools actually helped their students cheat on standardized tests. Standardized tests themselves aren't the problem; we've had them for decades, and they can provide valuable insights in some cases. The problem is when we focus on standardized tests to the exclusion of all else, putting a perverse pressure on schools. I believe you previously supported evaluating by standardized tests because they're the only quantitative measure we have; maybe this means we can't reliably quantitatively evaluate the impact of a single teacher on a small set of heterogeneous subjects. Demonstrate we can, then I'll reconsider NCLB.

edit to add another thought: I'd liken this to the problem of overfitting in machine learning. You have a way of measuring the performance of your system, and try to improve the system via that feedback. It's easy to overfit it, making it work beautifully on the provided data, but lack generality. The problem isn't in how we measure "fitness," necessarily, it is in how we update the system based on that result. That is, standardized tests may be a fine measure among students with that "generalized understanding"; I've not been convinced we are using those results intelligently.

Thanks for responding. I appreciate you taking the time to explain what you mean. I didn't mean to imply you criticized a specific test, I'm just trying to understand what critics of testing actually mean by "teaching to the test".

I suppose our probable point of disagreement is fundamentally that I don't think classes will ever impart a "general understanding". I consider the purpose of a class to be imparting a specific skillset, and most of the skillsets taught in school are quite amenable to testing.

You are also correct that my arguments in favor of testing implicitly assume teacher quality matters in a significant enough way to measure - if it doesn't, measuring them is pointless. (Then again, so is trying to use teacher quality as a lever to improve outcomes.)