News for hackers. Stuff that matters.
Re:Towards Understanding Sycophancy in Language Models by PaulHoule on Wednesday October 25, 2023 @02:54PM (#38016761)
It's part of their seductive ability that makes people think the glass is half full instead of half empty, why some people seem to think ChatGPT-4 can program better than they can while scientific evaluation shows that it's like: writes a program 80% of the time, the program compiles 20% of the time, the program works 5% of the time.

I can only imagine that somebody with poor judgement like Elon Musk has to have some people around who tell them what they want to hear and suck millions and millions from them.

Re:Towards Understanding Sycophancy in Language Models by pabo on Wednesday October 25, 2023 @03:31PM (#38017171)
> scientific evaluation shows that it's like: writes a program 80% of the time, the program compiles 20% of the time, the program works 5% of the time.

Can you share the source for this?

Re:Towards Understanding Sycophancy in Language Models by jiggawatts on Wednesday October 25, 2023 @05:33PM (#38018690)
There’s no source because he just made it up. Oh sorry, I meant that he hallucinated it. That’s the term we’re using now, isn’t it?
Re:Towards Understanding Sycophancy in Language Models by PaulHoule on Wednesday October 25, 2023 @06:26PM (#38019135)
I could swear I saw this the other day but I couldn’t find it in YOShInOn, here is an older paper which gets results like this for C/C++ but does a lot better with R, see Figure 1

https://arxiv.org/pdf/2308.04477.pdf

The results of this kind of eval could be across the board, you could pick out a set of examples worse than what I said (Games, C++) or pick one out that is really good (Algorithms, Ruby)

There was this one also

https://arxiv.org/abs/2310.12357

which showed some pitfalls…. In some case the LLM could say which project the source code was from which meant it had seen it in the training data and the code ought not to be in the test data.

Re:Towards Understanding Sycophancy in Language Models by j4yav on Wednesday October 25, 2023 @03:13PM (#38016961)
Copilot is great for explaining a bit of code, or suggesting an approach especially with a language I'm less familiar with. But, if I committed what it made, my code would turn into unmaintainable spaghetti.
Re:Towards Understanding Sycophancy in Language Models by readthenotes1 on Wednesday October 25, 2023 @03:14PM (#38016974)
The glass is always full. Just depends on what you value. Here on earth, water is often more important than air...
Re:Towards Understanding Sycophancy in Language Models by smcin on Wednesday October 25, 2023 @04:01PM (#38017526)
Retest that hypothesis in orbit...
Re:Towards Understanding Sycophancy in Language Models by a1369209993 on Wednesday October 25, 2023 @05:57PM (#38018913)
If you don't have enough reaction mass (for which water sometimes qualifies) to get out of orbit, having air but no water (rather than water but no air) just means you die three or four orders of magnitude more slowly. Maybe someone can lauch a rescue mission within a matter of days, but given how slow most space programs move these days, I'm not optimistic.
Re:Towards Understanding Sycophancy in Language Models by smcin on Wednesday October 25, 2023 @08:42PM (#38020253)
So, do we have a replication crisis?
Re:Towards Understanding Sycophancy in Language Models by coldtea on Wednesday October 25, 2023 @05:17PM (#38018524)
>while scientific evaluation shows that it's like: writes a program 80% of the time, the program compiles 20% of the time, the program works 5% of the time.

Where was this "scientific evaluation" published, on what LLM, and what size programs did it concerne?