News for hackers. Stuff that matters.
Re:A GPT-4 capability forecasting challenge by minihat on Saturday September 02, 2023 @10:30AM (#37361843)
My mental model of gpt-4 is apparently well calibrated for whether the model will give me a useful output that is close to what I asked for.

However, I'm not great at predicting whether the model will output a 100% correct response with no flaws whatsoever.

Unfortunately, this website mostly tests for the latter.

Re:A GPT-4 capability forecasting challenge by whimsicalism on Saturday September 02, 2023 @11:28AM (#37362364)
To me it is fascinating how when people are not super good at something, they often invent some secondary “true”/“better” task that they were actually good at
Re:A GPT-4 capability forecasting challenge by FabHK on Saturday September 02, 2023 @05:19PM (#37365618)
The website specifies its criteria for accepting an answer. Just use that threshold instead of whatever you in your mind deem “useful”.