News for hackers. Stuff that matters.
Re:Most leading chatbots routinely exaggerate science findings by bethekidyouwant on Monday May 26, 2025 @07:15PM (#44102489)
Most leading chatbots routinely exaggerate science findings - ahh yes this is completely unique to llms
Re:Most leading chatbots routinely exaggerate science findings by kjfaejgoaeijhei on Monday May 26, 2025 @07:18PM (#44102509)
To quote the article: "Chatbots were nearly five times more likely to produce broad generalisations than their human counterparts."

The following might be rude and unkind, but at this point frankly necessary: LLM apologists should stop projecting.

Re:Most leading chatbots routinely exaggerate science findings by ekianjo on Monday May 26, 2025 @07:27PM (#44102577)
> Chatbots were nearly five times more likely to produce broad generalisations than their human counterparts

Never seen a journalist not make broad generalizations on a scientific study so I am not sure where these numbers are coming from.

Re:Most leading chatbots routinely exaggerate science findings by kjfaejgoaeijhei on Monday May 26, 2025 @07:30PM (#44102598)
You can read the original research article here: https://royalsocietypublishing.org/doi/10.1098/rsos.241776

It will tell you exactly where the numbers are coming from - the comparison is between (a variety of) LLMs and "expert-written summaries from NEJM Journal Watch".

Re:Most leading chatbots routinely exaggerate science findings by redcobra762 on Monday May 26, 2025 @07:21PM (#44102529)
What is the human counterpart to an LLM? The article doesn't describe.

And "LLM apologists" is so polemical it's hard to take seriously. We get it, you don't like GenAI. That's fine, but can we talk about it without getting normative?

Re:Most leading chatbots routinely exaggerate science findings by kjfaejgoaeijhei on Monday May 26, 2025 @07:24PM (#44102565)
The article doesn't, but you can go to the "more information" section and keep digging:

"To systematically assess differences between LLM-generated and human-written summaries, we also collected the corresponding expert-written summaries from NEJM Journal Watch (henceforth ‘NEJM JW’)"

Re:Most leading chatbots routinely exaggerate science findings by bethekidyouwant on Tuesday June 03, 2025 @10:43AM (#44170672)
Right so not scientific American how convenient (and may I add scientific!)