Re:Re-Evaluating GPT-4's Bar Exam Performance
by mensetmanusman on Monday June 03, 2024 @01:11PM
(#40564765)
LLMs are not good at answering expert level questions at the forefront of human knowledge.
Re:Re-Evaluating GPT-4's Bar Exam Performance
by Zenzero on Monday June 03, 2024 @03:19PM
(#40566448)
Unfortunately it would be considered basic medicine in this case.
Re:Re-Evaluating GPT-4's Bar Exam Performance
by mensetmanusman on Monday June 03, 2024 @04:08PM
(#40566948)
Is it basic but not documented? Basic to me means the first google search result is generally correct.
Re:Re-Evaluating GPT-4's Bar Exam Performance
by Zenzero on Monday June 03, 2024 @06:46PM
(#40568525)
That's not how medicine operates.
Medical problems are highly contextual, so you are not going to get much valuable information at the level of what a doctor is thinking from the first page of Google. That doesn't mean it isn't a simple within our area of expertise.
Re:Re-Evaluating GPT-4's Bar Exam Performance
by mensetmanusman on Wednesday June 05, 2024 @08:17AM
(#40583959)
In my area of expertise, a well formulated google search can result in a page 1 full of academic articles on the general topic, but there isn’t necessarily consensus. This might be a case of the curse of knowledge :)
Try getting it to properly select crystalloids with proper additives for a patient with a given history and lab results and watch in horror as it confidently gives instructions that would kill the patient.
What is even more irritating is that I had gpt4 debate me on things that it was completely wrong about and it was only when I responded with a stern rebuke that it hit me with the usual "Apologies for the misunderstanding..."