News for hackers. Stuff that matters.
Re:GPT-4 identifies SVB’s biggest risk & gives good advice u... by bdg on Saturday March 25, 2023 @02:43PM (#35305698)
A problem I have with this type of experiment is that it's almost pre-messaging the bot what to say. Simply filtering down to very specific variables leads the outcomes in a very strong way. I suspect this exact approach will not work well for finding future problems.

I would be more interested if the variables were not pre-selected with the knowledge of what was important and could determine things that haven't happened, not because of the prediction but because this means it would understand how to pick the variables that are relevant (like the human did in this experiment).

Re:GPT-4 identifies SVB’s biggest risk & gives good advice u... by czx4f4bd on Saturday March 25, 2023 @03:19PM (#35306095)
Anecdotally, the times I've felt most impressed by ChatGPT usually ended up being exactly like this.

Even as an LLM skeptic, I've had chat sessions with ChatGPT that felt unbelievable at the time, where it almost seemed to be generating incredibly cogent ideas and arguments all on its own. Then, after a few days, I went and reread the transcript, only to realize I'd been hinting and nudging it along much more than I consciously thought at the time. I only remembered the answers that were the most impressive and forgot about the ones that missed the mark.

ChatGPT is really good at picking out the key ideas in the prompt and responding to them. It's really easy to inadvertently nudge it to give a certain type of response, which of course makes it feel all the more astounding when it gives you exactly the type of answer you already unconsciously expect.

Re:GPT-4 identifies SVB’s biggest risk & gives good advice u... by alpaca128 on Saturday March 25, 2023 @03:35PM (#35306306)
The AI equivalent of cold reading - just like with "psychics" where the audience wants to believe in the claimed abilities.
Re:GPT-4 identifies SVB’s biggest risk & gives good advice u... by czx4f4bd on Saturday March 25, 2023 @05:09PM (#35307378)
You must be psychic. I was literally thinking of cold reading as I wrote my comment.
Re:GPT-4 identifies SVB’s biggest risk & gives good advice u... by ASalazarMX on Saturday March 25, 2023 @08:50PM (#35309349)
It helps to think of it as augmentation rather than a separate entity. That way you can lead it consciously.
Re:GPT-4 identifies SVB’s biggest risk & gives good advice u... by dmreedy on Saturday March 25, 2023 @04:03PM (#35306687)
Indeed; I've been trying to describe these models as being consummate bullshitters. Or consummate improvisers, if you want to be kinder. Since its core task model is "autocomplete", it's less "trying to answer a question" than it is "trying to predict the most likely next thing to say after a question". Doing its best to sound like the most likely thing that comes after the thing prompting it, which in this case, often looks like an 'answer'. And when you have access to as much context as these models do, that can get as specific and nuanced as "giving you the answer you were expecting".

It has got me wondering if this might be an interesting fundamental model of consciousness. A bullshit generator modulated by other systems that may have more "logical" qualities.

Re:GPT-4 identifies SVB’s biggest risk & gives good advice u... by bdg on Saturday March 25, 2023 @04:22PM (#35306904)
I discuss this topic briefly in a post I recently made where I call it "decorative knowledge" and try to pinpoint exactly what GPT is doing and fails to do.

https://buildingbetterteams.de/profiles/brian-graham/navigat...

Re:GPT-4 identifies SVB’s biggest risk & gives good advice u... by tpoacher on Saturday March 25, 2023 @08:02PM (#35308954)
So ChatGPT is Clever Hans
Re:GPT-4 identifies SVB’s biggest risk & gives good advice u... by Borrible on Sunday March 26, 2023 @11:10PM (#35322453)
Talking to yourself is good for you as long as you know you are talking to yourself. You're training the social world model in your mind.

Eliza has come a long way, as Professors Higgins and Weizenbaum agree.

Re:GPT-4 identifies SVB’s biggest risk & gives good advice u... by nopinsight on Saturday March 25, 2023 @02:59PM (#35305892)
Perhaps someone could try other variations of the experiment as suggested above and by many here.
Re:GPT-4 identifies SVB’s biggest risk & gives good advice u... by sillysaurusx on Saturday March 25, 2023 @02:46PM (#35305734)
Could you give an example of what other variables could’ve been included?
Re:GPT-4 identifies SVB’s biggest risk & gives good advice u... by bdg on Saturday March 25, 2023 @02:48PM (#35305757)
Elsewhere in this discussion thread there's a few examples from im3w1l who saw this problem too:

> Like it could have mentioned jobs numbers but it didn't. It could have mentioned covid hospitalization statistics but it didn't etc

Re:GPT-4 identifies SVB’s biggest risk & gives good advice u... by LanceH on Saturday March 25, 2023 @06:15PM (#35308027)
We should be running it on 1000 banks right now, and post the result next year.
Re:GPT-4 identifies SVB’s biggest risk & gives good advice u... by lamontcg on Sunday March 26, 2023 @01:35AM (#35311362)
Yeah my immediate response to reading this was "objection! leading the witness!"

You know it failed due to interest rate risk and you prompt it to score about interest rate risk.

It turns out that the bank that failed due to interest rate risk scored highly on it.

If you already knew to ask that question back in 2021, you would have known the answer as well, but neither you or the LLM are predicting the future accurately.