I recognize it's not feasible for a class of students to constantly interrupt a lecture. The questions that other students had were never the same ones that I had, either. The problem is indeed scale, but I don't think motivation is a primary factor. Eventually you learn to be quiet and accept what the teacher and the textbook say.
Hear, hear! When people say that with better technology we could automate away teachers, I say that with better technology we could automate most everything else and spend the time mentoring one another.
Goodhart's Law (Popular formulation): When a measure becomes a target, it ceases to be a good measure.
Part of the problem is the lack of time. Both teachers and students need to cover all the topics in one semester/year, and it takes time to assimilate and integrate the knowledge. Add to that that as student, you don't know what's important, even when you're told so, if you're told at all. That's why we can hold contradictory models in our minds. They're good enough to solve some problems, but conceptually wrong.
I think that a possible solution would be to keep the grades open. You get a B in physics-101, since you got a flawed model in your mind, you got to physics-201 where new experiences show the problems in your model and force you to correct it, and now you prove you achieved a knowledge that deserves an A.
I know, that proposal it's impractical, and time consuming, but I think that's how most of us really learn.
I'd actually take the lesson the other way. It's hilariously tough for model-makers to acknowledge real-world noise that prevents their theoretically precise models from delivering reliable results.
https://v.cx/2010/04/feynman-brazil-education
> There are no experimental results mentioned anywhere in this book, except in one place where there is a ball, rolling down an inclined plane, in which it says how far the ball got after one second, two seconds, three seconds, and so on.
> The numbers have ‘errors’ in them – that is, if you look at them, you think you’re looking at experimental results, because the numbers are a little above, or a little below, the theoretical values. The book even talks about having to correct the experimental errors – very fine.
> The trouble is, when you calculate the value of the acceleration constant from these values, you get the right answer. But a ball rolling down an inclined plane, if it is actually done, has an inertia to get it to turn, and will, if you do the experiment, produce five-sevenths of the right answer, because of the extra energy needed to go into the rotation of the ball.
> Therefore this single example of experimental ‘results’ is obtained from a fake experiment. Nobody had rolled such a ball, or they would never have gotten those results!
> “If I ask you a question during the lecture, afterwards everybody will be telling me, ‘What are you wasting our time for in the class? We’re trying to learn something. And you’re stopping him by asking a question’.”
In my experience there are almost just 2 types of motivated students: those who want to understand the content of the course and those who want a good grade and ultimately graduate.
There's often a conflict of interest between them. For example the first group wants to learn more, ask more questions, go deeper. The second group sees this as a threat because the additional content may now be on the exam as well. And they often misunderstand this behavior of the first group as just being teacher pleasers/pets.
Or from own experience: when you tell the prof that he accidentally started to cover the same topic that we already covered in the last lecture (because you're there to learn more), you will annoy a lot of people who hoped to get one fewer lectures to study in the course. Less content, less effort, same grade.
You hear the first group having passionate discussions on physics or programming over a beer, while the other group discusses all the latest info of which prof is how demanding, what the grading criteria are, what the deadlines are, which departments you should or should not take courses from, where they have mandatory presence, who does multiple choice tests and who does free-text exams.
I don't have much to conclude, just an observation.
One of the things I think was the best ideas I learned in physics is that physics is all about approximations. Theorists tend to forget this annoyingly, but it helps keep naive scientists from using words like "fundamental" without qualifications, or saying garbage statements like "Newton's law X is wrong."
F = G * m1 * m2 / r²
This tells you that the acceleration of mass 1 is independent of the mass 1 and only depends on mass 2 and the acceleration of mass 2 is independent of mass 2 and only depends on mass 1. But the relative velocities will increase when either m1 or m2 is increased.
E.g. a black hole with the mass of the earth will hit the ground of a (perfectly rigid) earth in almost half the time as a feather would.
(I won't go into relativistic effects here.)
That is, would you expect adding ten pounds to have a meaningful difference in how fast a bike will accelerate down a hill? Even fifty pounds?
It will impact brake force needed to stop. But the speeds are going to be near identical.
I should clarify the last time I had this conversation, I'm talking about 40+ year olds building bikes. They were thinking a heavier bike would accelerate faster down a hill. That just is not how that works.
That is to say, the force from the air does not actively slow you down. It prevents you from speeding up, after a point. (Aerodynamics are, of course, more complicated. But this general model is pretty solid.)
Such that if I want to accelerate faster down the hill, I have to pedal to cause that to happen. Or add a motor.
Yes, acceleration due to gravity is the same, but air resistance has less effect, at all speeds, on the heavier bike. The rate at which the heavier bike gains speed is higher. It accelerates faster.
Take my statements to be, it won't hit a meaningful higher acceleration. Not that it won't necessarily accelerate for longer.
In general, you will accelerate down the hill at 9.8 m/s^2 modified by the incline. No matter how heavy the bike is. (Within the realm of realistic weights.).
Yes, the points you are raising are true. But within the realm of the biker and realistic bikes, not really relevant.
You can say that two larger things attract each other more than smaller things. Especially as they get closer to each other. Catch, of course, is all things on the earth are in the noise range compared to, you know, the earth. If you are talking about gravity and acceleration on the earth, you are talking about 9.8 m/s^2. Pretty much period.
And yes, there are terminal velocities that impact maximum speed. Acceleration, though, is not velocity.
It's clear that there's more force on a heavier object. How much force do you have to apply to counteract a bucket full of rocks from falling to the ground, vs. an empty bucket? Your arm is certainly applying more force to the former.
So then the question becomes Why doesn't the full bucket fall faster? Because it has more inertia, and thus it's more difficult to accelerate. But there's also more force, which overcomes this difficulty. The amount of greater force is exactly identical to the greater inertia, so the acceleration remains the same. The "exactly identical" here is no coincidence.
So yeah, that makes sense.
Similarly, I only mentioned terminal velocity because I said faster. Which could apply there. But my point was focused on acceleration of falling objects. I get that there are a lot of ways to muddy that discussion. As exemplified by my postings. :).
You can use a bunch of words to explain what's really going on there, but it won't change that the brick hits the earth first, hence it falls faster. Unless you can predict the future correctly and communicate this simply to your peers, what's the point of being correct in the terminology?
My point was strictly that many people will falsely think gravity somehow meaningfully pulls lighter objects at a lower rate. This despite days off lessons that gravity on Earth is basically a constant.
That is to say, if your model of speed of things falling is hinged solely on gravity, it will not be predictive in the extremes.
Of course, most folks aren't playing in the extremes, so the simple model will be fairly predictive of everything they do actually toss around the room. Can easily explain why the stack of books fell at the same speed as the basketball when the dresser was knocked over. :)
If only they had told me everything (including friction), then maybe phyics would have made a lot more sense to me. But then again, school math never even mentioned imaginary numbers. School is very incomplete.
Same goes for everything underwater. Which follows from viewing air as a medium you are traveling through.
Are there bad teachers that need to more honestly cover some of the assumptions? Yes. They are a useful tool, though.
To take your gravity example, the first question might be something about calculating gravitational force on the ISS. Then explain that it's rotating in space and therefore has centripetal force that accounts for this gravity. Then compute how fast it must be rotating. And from there you can switch to a completely different topic in the same context. Maybe something about EM radiation from the Sun that falls on the space station.
Shouldn't that be a sign that trying to come up with standardized measures is a fool's errand?
Short of blatant social engineering, I don't see how we can have a system that could improve people's motivation and drive for education. And if we don't have a way to actually change human nature, why do we keep trying to design this system that would magically work for so many different people?
We could give more autonomy to individual schools or teachers in terms of designing tests, but then we'd be facing two problems. One, the grades given by different schools/teachers would not be directly comparable. An A in one school could be awarded only for exceptional circumstances, and in another just for above-local-average. Two, once you start using those grades to judge people in a competitive situation like the job market, you'll have schools selling grades, teachers inflating grades to help their pupils compete against those from schools that sell grades, etc. and the whole system will collapse to uselessness.
Standardized testing mitigates these problems by forcing everyone to grade by the same criteria, but in order to do that, the grading must be done on things that are extremely easy to measure - which removes both individual judgement of the teachers and any relation between testing and understanding of the material.
The more I think about it, the more I feel it all boils down to scale. The grading system needs to be scaled up with the size of the market people compete on, otherwise it'll be either trivially gamed, or deeply unfair to unlucky people.
Accounting and law firms are looking for people with the stamina and single-minded goal focus to get document-creation tasks done, one after another, to a high degree of "skill." Getting an A on the test and preparing an SEC-compliant bond prospectus are close cousins of each other.
In fields where real-world mastery doesn't have much to do with the orderly routines of getting a 97 on the midterm, employers are less likely to hunt for people who got 97s on the midterm. (John Steinbeck stumbled around Stanford for six years and never graduated. But he got more accomplished as a writer than a whole decade worth of creative-writing majors with high GPAs.)
Are they really, or is just this elaborate illusion supported by most decision makers to give some rationalization to their own selection processes? When you have Harvard being sued for discriminating against Asians, how can we keep a straight face and tell the kids that they need to study hard because the grades matter?
I think this is part of what PG wants to defend in the essay: why do we actually care about grades at all?
Because jobs are at a premium and evaluating prospective hires is hard, so employers look for anything they could use to filter and sort the stream of candidates. Education - both the what and the where is somewhat correlated, so that's what's being used.
I think I spent too much on this discussion already, but the point I am trying to make is that standard tests are bad both as a filter for selection and as predictor of performance.
While not necessarily agreeing, I do understand if someone says "I will hire someone good from a top-tier school vs a top student from a not-so-good university". The "what" and "where" are definitely useful data points to filter out candidates.
What I don't understand is any employer that says they will hire based on SAT, GPA or any variation of an IQ test. It seems to be so unreliable measures to the point of actually hindering the process.
Throughout my life and career I've seen so many statements of this nature from my peers: "the status quo will only ever be a 90% solution at best, therefore we should throw all of that away and just do everything on a case-by-case basis in isolation from one another, then we'd all live in an ideal meritocracy (or something)".
See also any discussion about flat management. "Explicit management hierarchies obviously aren't perfect, therefore let's get rid of them altogether and assume that everyone is awesome and will magically self-organize and do the right thing."
I guess the common theme is, someone is greatly overestimating their own rationality, and projects it onto other people, and ends up with an almost religious faith in the self-organizing competencies of unmanaged people, or the natural inclination of unregulated schools or industries to always do the right thing even when it's not in their own self-interest.
IME there are lots of fundamentally messy or imperfect things in life, where any choice you make has negative consequences. There's also a personality type that is repelled by this sort of messiness and will always be drawn towards throwing everything away and starting from scratch, often with a childlike belief that things they don't understand aren't important, and why can't everyone see how obvious and elegant their solution is? IME they're usually wrong about those situations, but I guess I'm still glad people less pessimistic than I am are tilting at those windmills.
The problem isn't hyperrationality per se; it's a lack of intellectual humility and self-awareness. The behavior you're talking about arises when someone looks at a problem, immediately wants to fix it and makes snap judgements to figure out how to do so. Instead, they should dispassionately consider 1) whether the problem is fixable in any meaningful sense, and 2) whether they have the requisite experience, skill and insight to solve it in five minutes of thought.
You could further distill this idea with the observation that a lot of people consider things too systematically, rather than holistically. "First principles" thinking is a powerful drug, which appears to crack open the world and solve every problem. But no problem-solving paradigm can deconstruct away unknown unknowns or inexperience.
What I am saying is that what you regard as a "90% solution at best" is actually not a solution at all. Standardized tests are terrible predictors of performance on actual jobs. If standardized tests (or IQ tests or anything that is supposed to be a proxy for measuring performance) shows time and again that it only manages to measure people that are good to "study for the test", then why do we keep trying to use them to measure performance? Is it because it "the only thing we have"? This seems like saying "I am lost in Rome but all I have is a map of Paris, let me use that anyway."
And no, I am not proposing some other solution to replace it.
I am not asking for any utopia. What I am asking is "Instead of setting ourselves for failure every time coming up with some scheme for testing performance, why can't we just use actual performance on the activity that we want to evaluate candidates"?
It works for people in sports. It works for trade professions. It works for traders. It works for performers, entertainers, even entrepreneurs.
Why can't we have an educational system that does not put people on tracks to mold them and instead just coach them to follow whatever path they seem fit?
Sports works, but only because sports are extremely popular, with tens of millions of people enthusiastically participating on a regular basis, and accessible to anybody no matter their age and income. This is not true for, say, working at a law firm. You can’t take the best clerk from a pool of a thousand kids who have been clerking since elementary school. It’s either who you know, or who got good grades and test scores.
One would argue that this is the current status.
How objective are, e.g, IQ tests when I can take two different tests in a span of two hours and get from 128 to 146?
How my grades in Computer Engineering can be used to predict my performance as an Engineer in a country where 85% of the graduates would work in non-technical and more profitable fields?
If a company looking for a Quant gets someone with a Master in Math and a good tracking record knocking on their door, would they stop and ask to see if they can get someone with a PhD instead?
You are right that "proper evaluation takes time". What I am saying is that standardized tests should not count as proper evaluation.
> How objective are, e.g, IQ tests when I can take two different tests in a span of two hours and get from 128 to 146?
You can't take two proper WAIS-IV tests in two hours. In fact, unless you have been part of an unusual psychology study, you've probably never taken a real IQ test in your life. There are a lot of gimmicky free things on the internet that call themselves IQ tests but aren't predictive of anything.
As for the IQ test - it wasn't online and it wasn't a study, as far as I know - but I can totally concede that it was not a "real" IQ test.
Anyway, the story happened over 20 years ago. Part of the driving exam in Brazil included a "neuro-psychological evaluation" and the doctor flagged me for some extra tests. From what I remember I went to the clinic the next week, did one test, did the "neuro-technical" exam and was told to wait for an IQ test. When I got the form, it was the same as I got before originally (different order of questions) Long story short I asked why I was re-taking the test and I thought I "failed" somehow. They actually said they were not aware of the first one and later on found it.
Now that I think about it, perhaps it was some weird psychology test? :)
There just has to be an immediate value in what you teach. If there isn't, there's no point in teaching it. Students will just forget it anyway after the test.
Standardized testing is good for:
- measuring system performance against set standards - giving basic insights into student's average level of knowledge of a subject (if you use a thorough theoretical framework to conceptualise learning outcomes and link test items to this) - maybe measuring school performance against standards - maybe measuring teacher performance against standards
But for students this individual measure of an exam is simply not a good measure of learning in any modern education system. Yes it is a useful measure in really bad systems where teachers are bad and teach wrong things, or where eg attendance and exam fraud are huge issues.
But modern pedagogy is not about teaching book knowledge which is easily measured. In any Western education system we are mostly past that point. It's about helping students discover own interests and abilities, develop competence and confidence, experiment, fail and try again. This learning is simply too individual and hard to measure for a standardised test.
The only option is formative assessment (essentially teachers grading students repeatedly throughout the year on things like effort, achievement of general learning aims, additional development, results produced, presentations, support to other students, etc - a bit how you would measure a good office worker). Or maybe let's do like Finland and Estonia and let's mostly eliminate the testing burden.
I did Physics 2 that way and really liked it.
https://en.wikipedia.org/wiki/Keller_Plan
(also, wow, I just got that Principle Skinner in the Simpsons doesn't accidentally have that name. Face to palm.)
The basic principle of education is to start where the student is and then lead them step-by-step to what you want them to understand. That being the case, I would say physics should start with student's intuitive understanding and then explain, like through examples they can understand, why some of their intuitions are mistaken. Maybe this could even be turned into Socratic questioning.
For example, it's long been known in the physics education research community that students come away from introductory courses with very little physical understanding, even if they can do the plug and chug problems on typical tests just fine. Students can all recite Newton's third law, but immediately afterward claim that when a truck hits a car, the truck exerts a bigger force. They know the law for the gravitational force, but can't explain what kept astronauts from falling off the moon, since "there's no gravity in space". Another common claim is that a table exerts no force on something sitting on it -- instead of "exerting a force" it's just "getting in the way".
For research purposes, we measure physical understanding using a battery of tests, such as the Force Concept Inventory, containing only simple conceptual questions with unambiguous answers. So then everybody asks: if ordinary tests are so hackable, why not just switch to these conceptual ones? But that wouldn't work. There are less than ~100 distinct FCI-style questions. If these conceptual tests were the norm, students would just memorize the answers and parrot them back, with a flimsy understanding that crumples the second any follow-up question is asked. It would be just the same problem as before, except they would be worse computationally, too. The FCI only works as long as it doesn't count for a grade.
The problem isn't tests, it's scale. If the people aren't motivated, any standardized measure will miss the mark -- even the entrepreneurship Paul Graham advocates for. God knows I've seen a lot of bullshit in that direction.