Re:Re-Evaluating GPT-4's Bar Exam Performance
by Jensson on Sunday June 02, 2024 @04:45PM
(#40557100)
Problem is the LLM memorized the countless examples you can find of old BAR questions using extreme amounts of compute at training time, they don't have that ability to digest a specific case due to both lack of data and it doesn't retrain for new questions.
A human that can digest the general law can also digest a special case, but that isn't true for an LLM.
A human that can digest the general law can also digest a special case, but that isn't true for an LLM.