News for hackers. Stuff that matters.

A comment on Gemini Ultra Released. Parent comment

Re:Gemini Ultra Released by QuantumGood on Thursday February 08, 2024 @06:13PM (#39309104)
That is not the Monty Hall problem, it is a trick question based on the Monty Hall problem. It's a reasonable test, and I see GPT-4 recognizes the problem AS WRITTEN, and perhaps "the Dumb Monty Hall problem" is some generally accepted standard that I haven't encountered before.

edit: "AS WRITTEN"

[ Parent | on HN ]
Re:Gemini Ultra Released by sahila on Thursday February 08, 2024 @06:17PM (#39309149)
"Understands" is too strong of a word, more that it recognizes the problem as written. Here's yet a slight variation - just as simple - but changed enough it now is wrong.

https://chat.openai.com/share/22a9027f-a2c1-428a-94a2-8fd918...

[ Parent | on HN ]
Re:Gemini Ultra Released by mckeed on Friday February 09, 2024 @10:47AM (#39315981)
That's not GPT-4
[ Parent | on HN ]
Re:Gemini Ultra Released by kevinmchugh on Friday February 09, 2024 @12:52AM (#39311820)
I saw it posted on Twitter some time last year. If LLMs are to be useful they should be capable of answering novel questions. This is only a trick question for an LLM. 2 of the 7 sentences plainly state the answer.
[ Parent | on HN ]
Re:Gemini Ultra Released by QuantumGood on Sunday February 11, 2024 @09:44PM (#39340830)
You make a good point, but I have seen humans stick to what they know and ignore incredibly obvious contradictions. And there are similar trick questions designed to fool humans. This, though, is one that most humans would not be fooled by, as you point out.
[ Parent | on HN ]