> grade-driven teaching

I've never understood the modern theory that one can master a subject yet not be able to answer questions about it.

/watches straw man fall over without putting up a fight.

Would you rather work with someone who added a major subsystem to one of your compilers for fun or someone who had never coded but could recite the ISO standard by heart? Who believes the former person “would not be able to answer questions about [the subject]”?

If measurements tend to become targets over time then the goal of “develop expertise” measured by “pass a test” shifts to a goal of “pass the test”, and the test is limited to what you can ask hundreds of students in a couple of hours, and they’ve seen practise papers and last year’s papers and you can’t talk to them, and you can’t see them work or the results of their work.

Is that then a good way, to develop and validate their expertise? Is it the best available way?

> Who believes the former person “would not be able to answer questions about [the subject]”?

All the people who believe that "teaching the test" is not real knowledge.

For example, would you hire a lawyer who flunked the bar? How about consult a physician who couldn't pass his medical boards? How about get on an airplane with a pilot who flunked flight school, but "really knows how to fly"?

And yes, there are bad tests. But there are also good tests, and they work. Personally, I've never encountered a person who mastered a topic but couldn't get a good grade in it.

Lawyers who lose cases, physicians who couldn't save their patients, and pilots who have been in plane crashes still get to work in their occupation after the fact. Experience rules over all.

In your examples all those people never got to practice their craft. Lawyers can't practice law without a bar, physicians can't perform surgery without a medical license, etc. But software development isn't like that. What if there was a surgeon who has completed 10,000 successful surgeries but never went through med school? What if there was a lawyer who won major cases with no law degree? Or a commercial pilot who successfully flew for years on a fraudulent license and only stopped flying because he got caught? (That last one actually happened a few months ago).

If you could choose, would't you pick the surgeon with a wildly successful track record over the recently graduated med school student who has never done a surgery outside of residency?

> pilots who have been in plane crashes still get to work in their occupation after the fact.

Most of them are too dead to continue to work, and the rest usually get fired if the crash was due to pilot error.

And no, I don't believe you're going to fly with a pilot who flunked out of flight school.

Notice that I said “pilots who have been in plane crashes” and not “pilots who caused plane crashes.” This goes along with the theme of the lawyer losing the case, the surgeon losing the patient, etc. Sometimes you can do everything right and still everything goes wrong. Which is to say that the experience gained from these events makes these people better practitioners; it doesn’t matter what school they went to or what test they took if they gained real world experience practicing their craft.

No, I wouldn’t fly with a pilot who flunked out of flight school, but I would certainly fly with a pilot who’s flown for 10 years without a license over a pilot with a brand new PPL and <2000 hours flight experience.

Maybe for better perspective we can try this exercise: let’s say there’s a world renowned brain surgeon in, say, Norway who has worked for decades and performed tens of thousands of surgeries. No mistakes! Tens of thousands of happy stories!

But wait. Our intrepid surgeon can’t practice in the US because she didn’t go to medical school here! If she wanted to operate on a patient in the US mainland, she’d lose to a doctor who just exited residency!

If you were the patient who needed brain surgery, who would you pick? The world renowned expert or the guy who just finished training? Your logic dictates that the guy with the credentials wins out for no reason other than they went through whatever process to obtain those credentials, regardless if there are better metrics or methods to judge occupational success by.

> Our intrepid surgeon can’t practice in the US because she didn’t go to medical school here!

That's not my position. My position is if he was that good, he would pass the medical examinations.

> pilot

I'm not a pilot, but my dad was, other family members are, and I worked at Boeing designing the 757. Actually flying an airplane is rather simple. Most of pilot training is about handling an emergency.

These days, emergencies in the air are very rare. You may never have one in a career. But if you do have one, trying to learn how to deal with it on the job is a good way to die.

If someone really did fly 10,000 hours, but still cannot pass the certification test, he's a fool and you're a fool to fly with him.

Also, please do not confuse "did not take the test" with "flunked the test".

P.S. I have some training materials my dad had to learn. You're not going to learn that stuff by just flying around. Much of it is things that are learned about the airplane by expert test pilots, like how fast can you fly it without tearing the airplane apart. Do you think a pilot ought to know that speed before he gets in the cockpit? How about questions like how much runway do you need with a specific amount of airplane gross weight and altitude of the runway? I remember getting on a plane in Colorado and the pilot threw our luggage out, saying it was too much weight for the altitude, and it would arrive with the next flight. Do you want to fly with the pilot who couldn't make such computations? It doan' matter how good at flying he is if the airplane won't lift off the runway.

I don't buy for a minute you'd choose a lawyer who didn't pass the bar, etc.

I would certainly choose a lawyer who didn’t pass the bar provided they have won 10,000 cases in the past. Although I would settle for 1000-2000, etc.

I don't believe such a lawyer couldn't pass the bar.

Take all 8 billion human shapes, four right angle corners and straight sides is the test, all sides the same length is expertise. You're saying that all squares are definitely rectangles, which I agree with. I'm saying all rectangles aren't necessarily squares. Then it feels like you're saying there aren't many more rectangles than squares, citing that there are unquestionably squares. Unstated I'm suggesting there are a lot more rectangles than squares, by Sturgeons Law 95% of tests are crud, so there are far more people who passed a test than there are people who passed a test and have expertise. There are so many people in IT with coworkers who passed MCSE or A+ tests but cannot do what the test said they can do that it has become a meme, for one example.

That 5% of tests are rigorous and accurately identify squares is not really something I question, but I might question that the people who pass them were "taught to take the test" and nothing else.

(Would you say 7 years of degree, medical school, hospital residency, rotation around departments, counts as "teaching to the test", i.e. "an unhealthy focus on excessive repetition of simple, isolated skills ("drill and kill") [which] limits the teacher's ability to foster a holistic understanding of the subject matter."? - from https://en.wikipedia.org/wiki/Teaching_to_the_test )

If you cannot answer questions about a subject then you haven't mastered it, holistic or not.

Again you're repeating the assertion "if you have expertise you can answer questions", "squares are rectangles", which isn't the point of contention.

It's the other way which is in question, "if you can answer questions, does that mean you definitely have expertise?". Is being able to answer questions the same as having expertise? Are there people who CAN answer questions but have no expertise? If so, how many?

Referencing your other pilot comment, is being able to say what action you should take in an emergency, the same thing as having the calmness and presence of mind and judgement to actually take that action in an emergency instead of locking up and forgetting and panicking? Certainly the pilot who does act well in an emergency could tell you about it, but can all the people who can tell you about it, do it?

Please read carefully. I never said passing a test is proof of mastery. I said failing the test means no mastery.

Putting it in terms of logic, (A implies B) does not mean that ((not A) implies (not B)).

You absolutely did say passing a test is proof of mastery, and you cited lawyers and doctors passing tests as evidence, and you said "there are bad tests, but there are good tests and they work", with emphasis.

I passed German at school, despite being unable to speak German. Anybody who spoke German could have tried to have a conversation with me and see that I couldn't, that would be a good way to judge my supposed skills. Any test which is not that, any test which is a proxy to that, or a way to judge that with less effort, is worse than doing that. Teaching me to answer the questions they were going to ask on German grammar, or recite the sentences they were going to expect me to recite, is even worse.

Of course people with absolutely no skill fail tests more than random, who would question that.

> For example, would you hire a lawyer who flunked the bar?

It's not about hiring a lawyer who flunked the bar. It's more like, how many lawyers are bad lawyers despite passing the bar.

I don't know about law specifically, but in my experience a lot of people who pass tests don't have deep understanding. Vice versa may also be true depending on the test.

It's about the virtual certainty that those who didn't pass the bar are bad lawyers.

I think you've got that backward. I think parent poster is complaining about teaching methods where the belief is that "answering questions" -> "mastery." "Grade-driven teaching" here refers more to the "all we're gonna do is give you homework and a test." It's like industrialization of teaching - no mentoring or individual conversation, just tests and reading material. When you've got 1 teacher and 20-40 kids, it's about all you can do.

Lots of passing of tests from short-term memorization that's then promptly forgotten.

What’s interesting to me is that I’m tue internet you’ve got loads of students and loads of more experienced medium people meeting on stackoverflow and forums and right there where there is individual connection and back and forth, the experts keep trying to silence it because they want only expert discussions about advanced topics.

Like the companies that will help you leave as a customer because at least you leave with a good feeling and might come back, versus the companies which make it hard so you hate them forever. Experts go round Internet forums with a “fuck off lazy noob” attitude then wonder why the forum is in decline and empty with only noobs traffic, and then blame it on the noobs, because they are overrunning the site and that must be driving the “good people” out.

Mentoring and individual communication happens all over, but it looks like “doing their homework for them” which is verboten.

If they don’t learn from your answers, you might, other readers might.

The fix for “eternal September” is not “duplicate question” it’s that the older questions and answers need to be deleted, forgotten. The important bit is the development of ideas in the questioner and answerer, not the production of paperwork.

I suspect there is room for a huge forum/site based around these ideas that would combine collaborative editing, live examples, questions, and push new people not on “how to ask a good question” but “how to engage in a back and forth discussion” in some way.

This has been a disjointed blog post in a comment.

I've forgotten the material from many courses I've taken. But I've also discovered that the knowledge quickly returns if I retake a course on the same material, far quicker than it took the first time. The knowledge is still there, it just needs some WD-40 to break it loose again.

And yes, doing column buckling problem sets in college enabled me to size the jackscrew for the 757, as it was a column buckling problem.

I don't know anything about column buckling, but if you'd cranked out a bunch of column buckling problem sets and not paid any attention to when/where it would be applicable, would it still have helped you later? If you had forgotten the exact technique, would you have been able to check a reference book anyway?

Do you think memorizing names and dates of people and events from World War I gives much mastery of history? Or would memorizing the names of standard library functions and the academic definition of `pointer` and `recursion` give you much mastery of C?

(The flip side of this is folks I've worked with who will immediately say "this is the Command Pattern" or such and jump to writing a bunch of code that either is massive overkill and/or just breaks down when the real world introduces some edge cases.)

I knew how the column buckling equations were derived, therefore I knew when they were applicable and when they weren't.

As for checking a reference, I knew that "column buckling" was a thing. Before I studied it, I didn't know the concept even existed, and would have no idea how to look up something I didn't even know existed or what it might be called.

> Do you think memorizing names and dates of people and events from World War I gives much mastery of history?

How are you going to understand WW1 without knowing who Ludendorff was? I know that if you don't know names and dates and events, you cannot have a mastery of history. You don't have any framework to hang analysis or understanding on.

> Or would memorizing the names of standard library functions

Nobody does that. But I will say that if you can't name any of them, I guarantee you don't know squat about C.

> and the academic definition of `pointer` and `recursion` give you much mastery of C?

I wonder how many C experts you know can't answer a question about what a pointer is.

Yep, a lot of mini-problems that are generally relevant. I also show "real-world" code for every concept. Then they can see how this is applied in the Standard Library or other popular Python libraries.