>LLMs are trained to imitate patterns of language, not to discover or verify truth. So, when asked to speak as an expert in an area where perceived experts have a widespread misconception, the LLM will parrot that misconception, adopting the register and vocabulary of experts.
>This is dangerous when people take that register and vocabulary as conferring authority, or the very nature of an LLM as conferring truth or impartiality, which it cannot. They then amplify and entrench the misconception.
Only distantly related to the subject, but it's in the general area of mental models. I suppose if you're trying to code a cliche, the LLM will help you get it right, and if you're trying to create something original, the LLM will help you get it wrong.
> The real source of our theories is conjecture, and the real source of our knowledge is conjecture alternating with criticism.
(This is rephrased Karl Popper, and Popper cites an intellectual lineage beginning somewhere around Parmenides.)
I see writing tests as a criticism of the code you wrote, which itself was a conjecture. Both are attempting to approach an explanation in your mind, some platonic idea that you think you are putting on paper. The code is an attempt to do so, the test is criticism from a different direction that you have done so.