News for hackers. Stuff that matters.
Re:Placing #1 in Advent of Code with GPT-3 by amelius on Sunday December 04, 2022 @05:45AM (#33852419)
Huh, the prompt doesn't have contradictions.
Re:Placing #1 in Advent of Code with GPT-3 by grogers on Sunday December 04, 2022 @07:41AM (#33853154)
More like ambiguity. One could easily expect the resulting output for foobar(9) to print:

  baloney
  yam
But GPT doesn't understand the difference
Re:Placing #1 in Advent of Code with GPT-3 by amelius on Sunday December 04, 2022 @07:52AM (#33853243)
Not really, because in that case you'd expect a clear specification of the order in which the words should appear, if they should be on the same line, etc.
Re:Placing #1 in Advent of Code with GPT-3 by grogers on Tuesday December 06, 2022 @07:18AM (#33878527)
1. Change ... so that foobar(9) instead prints "yam".

2. Change ... so that foobar(9) also prints "yam"

1 is not ambiguous, but 2 is. It could mean that it does it's original behavior and also does the new behavior. But it could mean like foobar(10), foobar(9) also prints only "yam" as you are interpreting it to be.

I could imagine that the whole point was to see what GPT-3 did with the ambiguity, and it didn't even acknowledge it. Given that the prompt is already ambiguous I'm not sure why you'd assume that there would have to be a single expectation of the order of the prints, either one should suffice.

Or maybe it was just a mistake in the prompt because English is hard.

Re:Placing #1 in Advent of Code with GPT-3 by derangedHorse on Sunday December 04, 2022 @07:53AM (#33853258)
> foobar(9) prints "baloney" and foobar(10) prints "yam"

> change the function foobar so that foobar(9) also prints "yam"

It interpreted the prompt in the same way that I would, making foobar(9) only print "yam", but I guess I can see the interpretation where someone might think the prompt was asking to print both. It brings awareness to the effect of cultural bias in the training set and how that might require some people with different ways of speaking to add more clarification to get the output they want.

Re:Placing #1 in Advent of Code with GPT-3 by dphidt on Sunday December 04, 2022 @01:07PM (#33856031)
It's also assumed that "i" is an integer, a contextually plausible but potentially bad assumption. By one interpretation of the prompt, foobar(9) should print "yam" but foobar(9.5) still "baloney", for example.