Telling GPT-4 you're scared or under pressure improves performance

Re:Telling GPT-4 you're scared or under pressure improves pe... by ethbr1 on Saturday November 04, 2023 @08:32AM (#38140453)

I'd phrase characterizing the reliability of out-of-sample performance a priori as impossible, but not necessarily automatically failing.

There may be a subtle correlation between properties needed to answer a specific out-of-sample request and in-sample features.

Unfortunately, prior to training/testing and without recognizing that correlation in the data set, I believe it's impossible to guarantee the model will include it. (Corrections welcome)

[ Parent | on HN ]

Re:Telling GPT-4 you're scared or under pressure improves pe... by intended on Saturday November 04, 2023 @08:56AM (#38140633)

In essence: “You cant know in advance how far the model can approximate semantic patterns”

So claiming that out-of-sample performance is a mirage, would be a bridge too far?

[ Parent | on HN ]

Re:Telling GPT-4 you're scared or under pressure improves pe... by ethbr1 on Saturday November 04, 2023 @09:16AM (#38140751)

Maybe "a mirage that might actually be true"? Which is a terrible thing to rely on! Unless it's usually true?

[ Parent | on HN ]

Re:Telling GPT-4 you're scared or under pressure improves pe... by intended on Saturday November 04, 2023 @01:05PM (#38142918)

That measurement is the core of my current tasks. If you don’t know the error rate - then what are you doing ?

[ Parent | on HN ]

Re:Telling GPT-4 you're scared or under pressure improves pe... by ethbr1 on Saturday November 04, 2023 @02:07PM (#38143614)

Delivering what some executive promised when they told investors 'the company is using AI.' /s

[ Parent | on HN ]

Re:Telling GPT-4 you're scared or under pressure improves pe... by intended on Saturday November 04, 2023 @03:23PM (#38144252)

A Virtual beer/poison of choice to you and mjburgess in this thread.

[ Parent | on HN ]