Re:GPT4 Can’t Ace MIT
by iudqnolq on Saturday June 17, 2023 @09:02PM
(#36376321)
In general with evaluations LLVMs I keep seeing issues that would be caught by a human carefully looking at the results. To a nonexpert who occasionally peeks at things it seems like having a small dataset that's just slightly too big for you to manually review is a bad practice.
It also seems like 100% accuracy should have raised red flags, especially if you know your dataset isn't perfectly cleaned.
It also seems like 100% accuracy should have raised red flags, especially if you know your dataset isn't perfectly cleaned.