News for hackers. Stuff that matters.

A comment on Notes on OpenAI's new o1 chain-of-thought models. Parent comment

Re:Notes on OpenAI's new o1 chain-of-thought models by ramraj07 on Friday September 13, 2024 @12:53AM (#41528114)
Many of my PhD and post doc colleagues who emigrated from Korea, China and India who didn’t have English as the medium of instruction would struggle with this question. They only recover when you give them a hint. They’re some of the smartest people in general. If you try to stop stumping these models with trick questions and ask it straightforward reasoning systems it is extremely performant (O1 is definitely a step up though not revolutionary in my testing).
[ Parent | on HN ]
Re:Notes on OpenAI's new o1 chain-of-thought models by maeil on Friday September 13, 2024 @02:48AM (#41528634)
I live in one of the countries you mentioned and just showed it to one of my friends who's a local who struggles with English. They had no problem concluding that the doctor was the child's dad. Full disclosure, they assumed the doctor was pretending to be the child's dad, which is also a perfectly sound answer.
[ Parent | on HN ]
Re:Notes on OpenAI's new o1 chain-of-thought models by djur on Friday September 13, 2024 @03:10AM (#41528753)
The claim was that "it knows english at or above a level equal to most fluent speakers". If the claim is that it's very good at producing reasonable responses to English text, posing "trick questions" like this would seem to be a fair test.
[ Parent | on HN ]
Re:Notes on OpenAI's new o1 chain-of-thought models by andreasmetsala on Friday September 13, 2024 @03:17AM (#41528794)
Does fluency in English make someone good at solving trick questions? I usually don’t even bother trying but mostly because trick questions don’t fit my definition of entertaining.
[ Parent | on HN ]
Re:Notes on OpenAI's new o1 chain-of-thought models by rdtsc on Friday September 13, 2024 @04:04AM (#41529044)
Fluency is a necessary but not the only prerequisite.

To be able to answer a trick question, it’s first necessary to understand the question.

[ Parent | on HN ]
Re:Notes on OpenAI's new o1 chain-of-thought models by accountnum on Friday September 13, 2024 @06:10AM (#41529731)
No, it's necessary to either know that it's a trick question or to have a feeling that it is based on context. The entire point of a question like that is to trick your understanding.

You're tricking the model because it has seen this specific trick question a million times and shortcuts to its memorized solution. Ask it literally any other question, it can be as subtle as you want it to be, and the model will pick up on the intent. As long as you don't try to mislead it.

I mean, I don't even get how anyone thinks this means literally anything. I can trick people who have never heard of the trick with the 7 wives and 7 bags and so on. That doesn't mean they didn't understand, they simply did what literally any human does, make predictions based on similar questions.

[ Parent | on HN ]
Re:Notes on OpenAI's new o1 chain-of-thought models by rdtsc on Friday September 13, 2024 @07:16AM (#41530090)
> I can trick people who have never heard of the trick with the 7 wives and 7 bags and so on. That doesn't mean they didn't understand

They could fail because they didn’t understand the language. Didn’t have a good memory to memorize all the steps, or couldn’t reason through it. We could pose more questions to probe which reason is more plausible.

[ Parent | on HN ]
Re:Notes on OpenAI's new o1 chain-of-thought models by accountnum on Friday September 13, 2024 @07:37AM (#41530211)
The trick with the 7 wives and 7 bags and so on is that no long reasoning is required. You just have to notice one part of the question that invalidates the rest and not shortcut to doing arithmetic because it looks like an arithmetic problem. There are dozens of trick questions like this and they don't test understanding, they exploit your tendency to predict intent.

But sure, we could ask more questions and that's what we should do. And if we do that with LLMs we can quickly see that when we leave the basin of the memorized answer by rephrasing the problem, the model solves it. And we would also see that we can ask billions of questions to the model, and the model understands us just fine.

[ Parent | on HN ]
Re:Notes on OpenAI's new o1 chain-of-thought models by j_maffe on Friday September 13, 2024 @07:05AM (#41530016)
It does mean something. It means that the model is still more on the memorization side than being able to independently evaluate a question separate from the body of knowledge it has amassed.
[ Parent | on HN ]
Re:Notes on OpenAI's new o1 chain-of-thought models by accountnum on Friday September 13, 2024 @07:30AM (#41530172)
No, that's not a conclusion we can draw, because there is nothing much more to do than memorize the answer to this specific trick question. That's why it's a trick question, it goes against expectations and therefore the generalized intuitions you have about the domain.

We can see that it doesn't memorize much at all by simply asking other questions that do require subtle understanding and generalization.

You could ask the model to walk you through an imaginary environment, describing your actions. Or you could simply talk to it, quickly noticing that for any longer conversation it becomes impossibly unlikely to be found in the training data.

[ Parent | on HN ]
Re:Notes on OpenAI's new o1 chain-of-thought models by KoolKat23 on Friday September 13, 2024 @07:31AM (#41530177)
If you read into the thinking of the above example it wonders whether it is some sort of trick question. Hardly memorization.
[ Parent | on HN ]
Re:Notes on OpenAI's new o1 chain-of-thought models by lupire on Friday September 13, 2024 @03:33PM (#41534382)
Some people solve trick questions easily simply because they are slow thinkers who pay attention to every question, even non-trick questions, and don't fast-path the answer based on its similarity to a past question.

Interestingly, people who make bad fast-path answers often call these people stupid.

[ Parent | on HN ]
Re:Notes on OpenAI's new o1 chain-of-thought models by KoolKat23 on Friday September 13, 2024 @07:29AM (#41530167)
It's knowledge is broad and general, it does not have insight into the specifics of a person's discussion style, there are many humans that struggle with distinguishing sarcasm for instance. Hard to fault it for not being in alignment with the speaker and their strangely phrased riddle.

It answers better when told "solve the below riddle".

[ Parent | on HN ]
Re:Notes on OpenAI's new o1 chain-of-thought models by joedwin on Friday September 13, 2024 @01:58AM (#41528368)
lol, I am neither a PhD nor a postdoc, but I am from India . I could understand the problem.
[ Parent | on HN ]
Re:Notes on OpenAI's new o1 chain-of-thought models by ramraj07 on Friday September 13, 2024 @02:28AM (#41528535)
Did you have English as your medium of instruction? If yes, do you see the irony that you also couldn’t read two sentences and see the facts straight?
[ Parent | on HN ]
Re:Notes on OpenAI's new o1 chain-of-thought models by multjoy on Friday September 13, 2024 @04:41AM (#41529250)
“Don’t be mean to LLMs, it isn’t their fault that they’re not actually intelligent”
[ Parent | on HN ]
Re:Notes on OpenAI's new o1 chain-of-thought models by K0balt on Friday September 13, 2024 @06:43AM (#41529893)
In general LLMs seem to function more reliably when you use pleasant language and good manners with them. I assume this is because because the same bias also shows up in the training data.
[ Parent | on HN ]
Re:Notes on OpenAI's new o1 chain-of-thought models by lupire on Friday September 13, 2024 @03:39PM (#41534442)
"Don't anthropomorphize LLMs. They're hallucinating when they say they love that."
[ Parent | on HN ]
Re:Notes on OpenAI's new o1 chain-of-thought models by raincole on Friday September 13, 2024 @03:44PM (#41534480)
I think you have particularly dumb colleagues then. If you post this question to an average STEM PhD in China (not even from China. In China) they'll get it right.

This question is the "unmisleading" version of a very common misleading question about sexism. ChatGPT learned the original, misleading version too well that it can't answer the unmisleading version.

Humans who don't have the original version ingrained in their brains will answer it with ease. It's not even a tricky question to humans.

[ Parent | on HN ]
Re:Notes on OpenAI's new o1 chain-of-thought models by fragmede on Friday September 13, 2024 @04:20PM (#41534739)
> it can't answer the unmisleading version.

Yes it can: https://chatgpt.com/share/66e3601f-4bec-8009-ac0c-57bfa4f059...

[ Parent | on HN ]