News for hackers. Stuff that matters.

A comment on Tell HN: You can apparently get GPT-3 to cough up its own source code.

Re:Tell HN: You can apparently get GPT-3 to cough up its own... by version_five on Monday December 05, 2022 @09:46PM (#33875262)
Why would its source code be part of the training corpus? Anything is possible, but it could just as easily be making this up

Edit: actually I want to expand a little bit. This model has had a lot of work put in to ensure "safety" but only in a shallow sense. Real failure modes, like being unable to tell if it is lying about stuff, are ignored. Whether or not it for some reason has it's own source code memorized as part of the weights (which would really be weird), this is definitely a failure because it provides output that cannot be trusted.

[ Parent | on HN ]
Re:Tell HN: You can apparently get GPT-3 to cough up its own... by jacquesm on Monday December 05, 2022 @10:01PM (#33875364)
It could be, but it appears to be consistent across several runs.
[ Parent | on HN ]
Re:Tell HN: You can apparently get GPT-3 to cough up its own... by orbital-decay on Monday December 05, 2022 @10:37PM (#33875602)
It also doesn't mean anything - could be just entirely made up stuff from some metastable point in its space it converged on for your question. The model is really large and coherent, but it's not capable of drawing any line between facts and fiction, which makes it a perfect bullshitter.

This model is unable to follow any sort of logic, so it usually takes any nonsense question at face value and tries to answer it honestly, even if it means inventing very plausible lies in the process. Unless it goes against its biases of course - in that case they take preference. This is enough for it to sound very confident, which can trick most people. (see also the warning in my profile)

[ Parent | on HN ]
Re:Tell HN: You can apparently get GPT-3 to cough up its own... by ugurnot on Monday December 05, 2022 @11:06PM (#33875796)
Having a consistent output to same input is something. I have done a couple of experiments starting with same sentence and each time it was a little bit nuanced. I would be interested if the answer is the same for a targeted question. Still, it would be better not to draw conclusions early.
[ Parent | on HN ]
Re:Tell HN: You can apparently get GPT-3 to cough up its own... by orbital-decay on Monday December 05, 2022 @11:15PM (#33875836)
Most likely their repo is similar to hundreds of others, so it probably just gives the structure of some typical ML code (which is imprinted in the model really well), skewed towards the initial request.
[ Parent | on HN ]