Sorry, but a new prompt for GPT-4 is not a paper

Re:Sorry, but a new prompt for GPT-4 is not a paper by aleph_minus_one on Tuesday December 05, 2023 @08:41AM (#38530548)

> People overestimate the value of "grand developments", and underestimate the value of actually knowing - in this case actually knowing how well something works, even if it is as simple as a prompt.

I think this depends a lot on the "culture" of the subject area. For example in mathematics, it is common that only new results that have been thoroughly worked through are typically "publish-worthy".

[ Parent | on HN ]

Re:Sorry, but a new prompt for GPT-4 is not a paper by brookst on Tuesday December 05, 2023 @08:52AM (#38530647)

Wouldn’t the “thoroughly worked through” part be analogous to extensive measurements of a prompt?

[ Parent | on HN ]

Re:Sorry, but a new prompt for GPT-4 is not a paper by aleph_minus_one on Tuesday December 05, 2023 @11:14AM (#38532774)

Let me put it this way: you can expect that a typical good math paper means working on the problem for, I would say, half a year (often much longer). I have a feeling that most papers that involve extensive measurements of prompts do not involve 1/2 to 1 year of careful

- hypothesis building

- experimental design

- doing experiments

- analyzing the experimental results

- doing new experiments

- analyzing in which sense the collected data support the hypothesis or not

- ...

work.

[ Parent | on HN ]