News for hackers. Stuff that matters.
Re:Gemini 2.0: our new AI model for the agentic era by CamperBob2 on Wednesday December 11, 2024 @03:14PM (#42392375)
I've started keeping an eye out for original brainteasers, just for that reason. GCHQ's Christmas puzzle just came out [1], and o1-pro got 6 out of 7 of them right. It took about 20 minutes in total.

I wasn't going to bother trying those because I was pretty sure it wouldn't get any of them, but decided to give it an easy one (#4) and was impressed at the CoT.

Meanwhile, Google's newest 2.0 Flash model went 0 for 7.

1: https://metro.co.uk/2024/12/11/gchq-christmas-puzzle-2024-re...

Re:Gemini 2.0: our new AI model for the agentic era by nrvn on Wednesday December 11, 2024 @04:19PM (#42393095)
Did it get the 8 right? The linked article provides the wrong answer btw.
Re:Gemini 2.0: our new AI model for the agentic era by CamperBob2 on Wednesday December 11, 2024 @07:01PM (#42394543)
I didn't see a straightforward way to submit the final problem, because I used different contexts for each of the 7 subproblems.

Given the right prompt, though, I'm sure it could handle the 'find the corresponding letter from the landmarks to form an anagram' part. That's easier than most of the other problems.

You're saying the ultimate answer isn't 'PROTECTING THE UNITED KINGDOM'?

Re:Gemini 2.0: our new AI model for the agentic era by nrvn on Thursday December 12, 2024 @05:59AM (#42398150)
if you follow the sleigh morse path starting from the robin it will be 'united in protecting the kingdom'.
Re:Gemini 2.0: our new AI model for the agentic era by p1esk on Wednesday December 11, 2024 @04:41PM (#42393292)
Wow! That’s all I need to know about Google’s model.
Re:Gemini 2.0: our new AI model for the agentic era by Workaccount2 on Wednesday December 11, 2024 @05:43PM (#42393879)
What is impressive about this new model is that it is the lightweight version (flash).

There will probably be a 2.0 pro (which will be 4o/sonnet class) and maybe an ultra (o1(?)/Opus).

Re:Gemini 2.0: our new AI model for the agentic era by danpalmer on Wednesday December 11, 2024 @05:09PM (#42393570)
That's a comparison of multiple GPT-4 models working together... against a single GPT-4 mini style model.
Re:Gemini 2.0: our new AI model for the agentic era by p1esk on Thursday December 12, 2024 @12:21AM (#42396458)
multiple GPT-4 models working together

What do you mean? Is o1 not a single model?

Re:Gemini 2.0: our new AI model for the agentic era by iamdelirium on Wednesday December 11, 2024 @04:42PM (#42393301)
Why are you comparing flash vs o1-pro, wouldn't a more fair comparison be flash vs mini?
Re:Gemini 2.0: our new AI model for the agentic era by iamdelirium on Wednesday December 11, 2024 @04:49PM (#42393368)
I just ask o1-mini the first two questions and it got it wrong.
Re:Gemini 2.0: our new AI model for the agentic era by CamperBob2 on Wednesday December 11, 2024 @06:57PM (#42394511)
It's the only Google model that my account has access to that accepts .PNG files. I assumed it was the latest/greatest experimental 2.0 release.

If they want a rematch, they'll need to bring their 'A' game next time, because o1-pro is crazy good.