> Paste your comment into chatgpt and see what it says.

Isn't one of the bigger problems with ChatGPT that it's much too supportive of whatever the human is talking about?

I guess it depends on how you frame it. "I've just posted this comment, what do you think" vs "Someone online has just posted this comment, what do you think".

But it does require to know the bias that LLMs have ahead of testing this.

Thats the point. If even such a sycophantic ai disagrees with your points you have a problem

I didn't read their whole comment, but I worked in the Internal Research department of a medical school. I did their statistical studies and built software for analysis pipelines.

Doctors, at least 15 years ago, were definitely bad at statistics.

They were not required to take a statistics course at all. Most programs would require Algebra and Calculus as part of their science reqs.

Some would maybe take one basic research course, and they would then become obsessed with p values of 0.05.

They did not have a basic understanding of how to interpret research unless they were an auto didactic and went out of their way to improve. It's something my director (a doctor and software engineer), and the Dean complained about relentlessly.

I’ve not only visited many doctors personally I was also part of a team working on a medical diagnostic instrument, the result of the instrument was a probability distribution function and it was impossible to explain this to the doctors who really would only accept a small number discrete classifications, which in effect throws out about half of the data we had worked so hard to collect.

Yes! good point, I have noticed this as well.

You reminded me about another idiosyncrasy: Doctors are addicted to double blind randomized control trials.

Which yes, those are powerful. But good evidence can come from many other study designs. Especially when mechanisms and first principles are being studied.