News for hackers. Stuff that matters.
Re:Compare how GPT-2, 3, 3.5 and 4 answer the same questions by jellyberg on Tuesday October 31, 2023 @04:24PM (#38090844)
Yeah it's hard to compare across models, interested in suggestions here.

We give all models a bunch of few-shot examples, which improves GPT-3 (davinci)'s question answering substantially. GPT-2 sometimes generates something that answers the question, sometimes it's just confused. Click "See full prompt" to see the few-shot examples that the models get.

Our goal was to exercise the full capabilities of each model.

There are no comments on this comment yet.