News for hackers. Stuff that matters.

A comment on GPT-4 and professional benchmarks: the wrong answer to the wrong question. Parent comment

Re:GPT-4 and professional benchmarks: the wrong answer to th... by User23 on Tuesday March 21, 2023 @09:42AM (#35245959)
The main problem with using GPT-3 (and maybe 4 I dunno) in that way is it will happily bullshit you to the point of making up fake references. For example it quoted me "Section 6.2" of the Go Programming Language Specification to support its answer when I asked it how a particular conversion is specified.
[ Parent | on HN ]
Re:GPT-4 and professional benchmarks: the wrong answer to th... by billythemaniam on Tuesday March 21, 2023 @10:36AM (#35246812)
So far GPT-4 seems to improve on this problem. Still happens but less frequent.
[ Parent | on HN ]
Re:GPT-4 and professional benchmarks: the wrong answer to th... by mtlmtlmtlmtl on Tuesday March 21, 2023 @11:00AM (#35247181)
I'm assuming you base this on hard empirical data and not just that it feels like it when you use it? ;)
[ Parent | on HN ]
Re:GPT-4 and professional benchmarks: the wrong answer to th... by dpkirchner on Tuesday March 21, 2023 @09:59AM (#35246187)
Do you remember the prompt (it should be in the history column)? I'm curious how it came up with the section numbering.
[ Parent | on HN ]