Re:Show HN: Automatic Prompt Optimizer for LLMs
by iudqnolq on Saturday April 22, 2023 @01:16AM
(#35663589)
> The prompt it generates looks good and makes sense
What I'm trying to say is that that's exactly what it's optimized for. They're predicting what sounds plausible based on all the pre-gpt writing about AI.
But GPT was revolutionary! A lot of the pre-gpt blogspam and reddit comments and fiction and so on was wrong about how AI works in exactly the way you've been socialized to find plausible.
In general plausibility is the wrong metric to evaluate GPT on, and it's wronger than it seems like it should be.
What I'm trying to say is that that's exactly what it's optimized for. They're predicting what sounds plausible based on all the pre-gpt writing about AI.
But GPT was revolutionary! A lot of the pre-gpt blogspam and reddit comments and fiction and so on was wrong about how AI works in exactly the way you've been socialized to find plausible.
In general plausibility is the wrong metric to evaluate GPT on, and it's wronger than it seems like it should be.