From GPT-4 to GPT-5: Measuring Progress in Medical Language Understanding [pdf]

Re:From GPT-4 to GPT-5: Measuring Progress in Medical Langua... by xnx on Thursday August 21, 2025 @07:59PM (#44979644)

Have you looked at comparing to Google's foundation models or specialty medical models like MedGemma (https://developers.google.com/health-ai-developer-foundation...)?

[ Parent | on HN ]

Re:From GPT-4 to GPT-5: Measuring Progress in Medical Langua... by fertrevino on Friday August 22, 2025 @03:59AM (#44981999)

That would be an interesting extension. MedGemma isn't part of the original benchmark either [1]. Since Gemini 2.0 Flash is on 6th place, expectations are for MedGemma to achieve higher than that :)

[1]https://crfm.stanford.edu/helm/medhelm/latest/#/leaderboard

[ Parent | on HN ]