News for hackers. Stuff that matters.
Re:Gemini Ultra Released by pierrebai on Thursday February 08, 2024 @11:25AM (#39303898)
You are right and I had the same reaction. The correct answer should have been: (f (h (g (h z)))) AKA ((1 + 5) * 2) + 5.

Is it not ironic that the supposedly test of AGI is flawed and its human designer fail to see it and denied it when presented with facts. Maybe the test designer is hallucinating just as much as those LLM? :)

Re:Gemini Ultra Released by scarmig on Thursday February 08, 2024 @12:03PM (#39304413)
> The correct answer should have been: (f (h (g (h z)))) AKA ((1 + 5) * 2) + 5.

Isn't that (h(g(h z)))?

And, FWIW, at least in my test, Gemini gets that in its final answer, though it failed in the two other drafts:

https://g.co/gemini/share/c922e7ef62aa

ChatGPT sputters:

https://chat.openai.com/share/25abbf47-2ed4-4635-a351-90a9a6...

(ETA more Gemini testing suggests its correct answer was a one-off)

Re:Gemini Ultra Released by mahogany on Thursday February 08, 2024 @11:29AM (#39303948)
It's pretty amusing and it is not the first time I've seen this. Random example: https://news.ycombinator.com/item?id=38387168

It's a little scary that it can be so hard to evaluate the correctness of these LLMs even when we are paying close attention and looking for mistakes. Or maybe the scary part is that we can become biased when we want to believe.

Re:Gemini Ultra Released by LightMachine on Thursday February 08, 2024 @11:33AM (#39304002)
Oh fuck. Well, in my defense, nobody is claiming I'll design fusion reactors and cure cancer by 2027
Re:Gemini Ultra Released by og_kalu on Thursday February 08, 2024 @04:44PM (#39308149)
No, but we at least acknowledge you a general intelligence like all humans. I'm not sure when agi - artificial general intelligence began to mean anything other than artificial and generally intelligent.

agi may as well be God, the bars some people have.