Even as an LLM skeptic, I've had chat sessions with ChatGPT that felt unbelievable at the time, where it almost seemed to be generating incredibly cogent ideas and arguments all on its own. Then, after a few days, I went and reread the transcript, only to realize I'd been hinting and nudging it along much more than I consciously thought at the time. I only remembered the answers that were the most impressive and forgot about the ones that missed the mark.
ChatGPT is really good at picking out the key ideas in the prompt and responding to them. It's really easy to inadvertently nudge it to give a certain type of response, which of course makes it feel all the more astounding when it gives you exactly the type of answer you already unconsciously expect.
It has got me wondering if this might be an interesting fundamental model of consciousness. A bullshit generator modulated by other systems that may have more "logical" qualities.
https://buildingbetterteams.de/profiles/brian-graham/navigat...
Eliza has come a long way, as Professors Higgins and Weizenbaum agree.
> Like it could have mentioned jobs numbers but it didn't. It could have mentioned covid hospitalization statistics but it didn't etc
You know it failed due to interest rate risk and you prompt it to score about interest rate risk.
It turns out that the bank that failed due to interest rate risk scored highly on it.
If you already knew to ask that question back in 2021, you would have known the answer as well, but neither you or the LLM are predicting the future accurately.
I would be more interested if the variables were not pre-selected with the knowledge of what was important and could determine things that haven't happened, not because of the prediction but because this means it would understand how to pick the variables that are relevant (like the human did in this experiment).