Re:Superhuman performance of an LLM on the reasoning tasks o...
by inopinatus on Thursday May 29, 2025 @07:18PM
(#44131309)
The situation is more akin to a much earlier situation in chess, from 1997, in that Deep Blue could only beat Kasparov with a dedicated team of IBM engineers and GM consultants revising the code between matches, and it still needed a human to interact with the actual chessboard.
We remain a very long way from “ChatGPT will see you now”.
In the meantime, in the real world, I suspect the infamous "Dr Google" is being supplanted by "Dr LLM". It will be difficult to ethically study whether even this leads to generally better patient outcomes.
_________
edit: clarity
Re:Superhuman performance of an LLM on the reasoning tasks o...
by htrp on Thursday May 29, 2025 @08:13PM
(#44131599)
> I suspect the infamous "Dr Google" is being supplanted by "Dr LLM".
Absolutely.
Re:Superhuman performance of an LLM on the reasoning tasks o...
by thbb123 on Thursday May 29, 2025 @07:06PM
(#44131257)
Algorithm aversion and automation biases have been thoroughly studied over the past 70 years of human factors for industrial security. All in all, the thought processes of humans are not always compatible with the evidence on which automation works.
Check out Fitts, HABA-MABA for more results.
1. ChatGPT O1 significantly outperformed any combination of Doctor + Resources (Median score of 86% vs 34%-42% of doctors). Hence superhuman results (at least compared against average physicians)
2. ChatGPT + Doctor performs worse than just ChatGPT alone.
This means that the situation is getting similar to Chess - where adding Magnus Carlsen as a helper to Stockfish (a strong open source chess enginer)) could only make Stockfish worse.