I had human teachers who did that in middle/high school. Took me many years to pick out all the hallucinated bits of "knowledge". I don't think the current models are any less reliable that what we currently have on average.

I'll always remember my middle school science teaching telling us that nuclear fusion violates conservation of mass because the 2 protons in a pair of hydrogen nuclei combine to make helium with 4 nucleons. It's not true, but that's not the point.

But he was a great teacher anyway. He was engaging and kept the kids in line and learning. I eventually learned the truth, and most of my classmates forgot about it. Teaching, like flying a plane or driving a train, might become more about keeping watch over a small group of people and ensuring that things don't go off the rails, and that's fine.

This one feels less sinister than some other things at least to me, personally. You can reasonably doubt that the conservation of mass is violated and find out the truth based on that. But understanding more complex biology or historical context for some things? Granted, many of these things seem to be low stakes, but I'm sure there are some there are not (sex ed comes to mind).

to be fair, fusion does violate conservation of mass, just not the way the teacher explained it. the loss of mass is where the energy comes from.

Yes, there is no law of conservation for mass like there is for energy. Fusion is a good example for why it's not conserved. The teacher was right.

He was right that it violates conservation of mass. He was completely wrong that it violated it by adding 2 atomic mass units when hydrogen fuses.

In reality heavier isotopes of hydrogen fuse, conserving the total number of nucleons, but the resulting hydrogen has a lower rest mass than the parent particles. The extra mass is released as energy and the total energy is conserved.

By his logic the system either violated energy conservation (by creating nucleons while releasing energy) or was endothermic (creating nucleons from the surrounding energy).

There actually is a law of conservation of mass (it's the same law, because mass is energy) and it only appears violated if you forget about the particles that are zooming away at the speed of light. Of course the mass of a system changes if mass can flow in and out.

Mass is not the same as energy. Mass can be converted to energy or has energy, but a photon, for example, is massless while carrying energy.

That is incorrect. Photons have mass. They have no rest mass. They also cannot rest, so you might wonder how relevant that is.

The concepts of rest mass and relativistic mass are considered outdated. In modern physics, "mass" means what they meant by "rest mass".

Here some indication I'm not making this up: https://hsm.stackexchange.com/questions/2465/when-and-why-di...

In any case, I never use those concepts, and I know no professional particle physicist that does. By "mass", I mean rest mass.

When you put a photon in a stationary box, the "relativistic mass" of the photon becomes part of the "rest mass" of the photon-box system. You can't ignore it.

Yes, together with mass-energy equivalency it would form a coherent argument, and then also a correct one - but the thing is that if incomplete, it still might sound funky enough to you to research it if you care.

I think it helps that it's a very narrow field to look at, compared to fuzzy and big-picture view of social studies, for example. So much room to be confidently wrong... And sadly I can't think of a solution, LLMs or not.

I had a chemistry teacher who told us that hydrogen reacts violently with oxygen, and this is how the hydrogen bomb works.

I had a chemistry teacher who insisted that the fissile isotope of Uranium was U-238 not U-235. I challenged him on this multiple times and he refused to budge on this. I get that it's a simple mistake to make (it seems like U-238 is bigger so intuitively ought to be less stable) but he could have just looked it up and he didn't, I guess he was just so confident about it that he thought there was no way he could have been wrong about it.

Hey it's a bomb made out of hydrogen! Also the deployment system for a thermonuclear bomb might involve that reaction in the rocket engine.

Well you can make a hydrogen "bomb" that way. Just not the hydrogen bomb.

I had one that mentioned this too :(

I mean fusion and fission do violate conservation of mass and conservation of energy, they just don't violate conservation of mass and energy, right? We thought mass was strictly conserved until Einstein, and then we updated our understanding.

That's an American problem though. In most of Europe you need a masters degree to teach highschool and that involves at least an undergrad level of understanding the subjects you will teach.

E.g. in Hungary I had a university CS professor that originally wanted to be a highschool teacher and a highschool physics teacher that originally wanted to be researcher. Their choice of degree didn't determine which outcome they got. The researcher and teacher curriculum had an 80%+ overlap.

I think it’s pretty common for states to require a masters degree to maintain your teachers certification.

You also have to pass a standardized test specifically on subject matter in order to get your teaching certificate.

The undergrad degree I did was split into thirds, one for subject matter, one for teaching pedagogy, and one for teaching your subject matter.

I think they are less reliable. For factually verifiable facts LLMs are doing worse than 90% for me. I've been told some incorrect things by educators, but at a much lower rate.

The problem is that people seem to trust whatever AI hallucinated way more than if they heard same thing from human

To be fair, that was much of my actual experience with human professors in university.

Veritasium proved that in a difficult challenge.

A Physics Prof Bet Me $10,000 I'm Wrong

https://www.youtube.com/watch?v=yCsgoLc_fzI

Yeah one of my teachers was able to identify which high school I had come from due to something I had been mistaught.

Off the top of my head: DOMS being little crystals in muscles, tongue having separate areas for each type of taste, food pyramid, blue blood in the veins, the appendix being useless, body temperature doesn't change disregarding whether it's exposed to cold or to heat, and a whole lot of stuff related to politics and history I'd rather just omit (I don't live in the US).

All things I learned in school which were wrong information.

Not to mention, the current state of education is far worse. I don't think most realize how low the bar is.

My biology teacher in school once tried to teach us that winds created by God. Not like spiritually or something but that God literally made the wind I guess.

My “earth sciences” teacher also once tried to argue with me against the universal law of gravitation. (no, she was not referring to Special/General Relativity. She didn’t agree two objects in a vacuum fall at the same speed regardless of mass.

One of my teachers in elementary school told us that people in the Arabic world wore long garments because as Muslims, they believed the Messiah would be born by a male, and thus, it was important to have something to catch the baby as it unexpectedly popped out one day and would otherwise hit the ground.

She only really had two faults: She wasn't very bright, and she wasn't fond of children. I had her in about 80% of all my classes for six years. High school was a relief.

It may interest you to know that this was a misremembered truth.

It is widely believed by their neighbors, that the _Druze_ wear baggy pants because they believe that the Mahdi will be born to a male, and the pants will catch the baby etc. I say "widely believed", the Druze are famously secretive and will not confirm or deny most things about their religion. The 'elect' Druze men do wear distinctive baggy trousers with the crotch down around the knees: no one else does.

The Druze are people in the Arabic world: moreover, they are Arabs. They began as an Isma'ili sect, but do not identify as Muslim: they call themselves al-Muwaḥḥidūn, meaning 'the monotheists', or 'unitarians'.

Much closer to correct than not!

Thank you, that is an interesting tidbit!

They'll also encourage and praise you even when you're heading down the wrong path until you think you've uncovered the secret of the universe or proven that established science was wrong this whole time when really you've just been bullshitting with an engagement bot.

No, they don't really do that anymore, if you use the latest models with reasoning enabled.

Like almost everything else about LLMs, this unfortunate tendency has gotten a lot better recently, which you might not realize if you gave up after getting some lame answers or bogus glazing on the free ChatGPT page a couple of years ago.

Anti-intellectualism is at it again, hu?

Like humans.

I think we should go a little deeper on this idea.

We can all agree that both human "experts" and LLMs can sometimes be right, and sometimes be confidently wrong.

But that doesn't imply that they're equally fit for purpose. It just means that we can't use that simple shortcut to conclude that one is inferior to the other.

So where do we go from here?

I’ve always thought of the definition of “expert” as reliably knowing the difference between what is known, what is speculated but unproven, and what is unknown. People claim expertise in all sorts of things that they aren’t experts in. But true experts should not be wrong. They should qualify levels of certainty. This definition certainly works in the sciences.

In reality few humans are true experts on every topic they open their mouth on. A high school teacher in science is hardly a true expert in every single thing they teach.

The amount of bullshit and blatant lies I’ve heard from my human teachers dwarfs the hallucinations produced by today’s LLMs.