News for hackers. Stuff that matters.
Re:Gemini Ultra Released by alphabetting on Thursday February 08, 2024 @08:13AM (#39301547)
That's a bummer. I just made this one up which GPT-4 failed and Gemini got right but definitely need to do more testing to see what's being taken from online and what is actual reasoning.

https://i.imgur.com/3sNr3LW.png https://i.imgur.com/EIj0nZg.png

Re:Gemini Ultra Released by nycdatasci on Thursday February 08, 2024 @08:42AM (#39301795)
Great example! Thanks for sharing. I tested the same prompt using a variety of GPT-4 model versions and it consistently got the answer wrong. When I continued with "use algebra/formulas to confirm your answer" it corrected the initial mistake, but I'm surprised that it's not doing that out of the box.
Re:Gemini Ultra Released by mewpmewp2 on Thursday February 08, 2024 @09:03AM (#39302002)
I tried through API with latest GPT-4-Turbo-Preview and this is what I got:

------

Let's break down the information given:

The school is two blocks north of the pool. This means if you are at the pool and move two blocks north, you'll arrive at the school.

The convenience store is one block south of the school. Therefore, if you start at the school and move one block south, you'll reach the convenience store.

Based on the first two statements, the relative positions from north to south are: School, Convenience Store, Pool.

Thus, the third statement saying "The pool is south of the convenience store" is indeed true, based on the positions provided by the first two statements.

-----

But it starts weirdly messing up, when I asked it to explain it using coordinates.

---

Let's use a simple coordinate system to represent the positions of the school, the pool, and the convenience store. Let's assume each block corresponds to one unit in this system. We'll start with the pool as our reference point at coordinates (0,0). Remember, moving north increases the y-coordinate, while moving south decreases it.

The school is two blocks north of the pool. If the pool is at (0,0), moving two blocks north puts the school at (0, +2).

The convenience store is one block south of the school.

If the school is at (0, +2), moving one block south puts the convenience store at (0, +1).

Given these positions:

The pool is at (0, 0).

The school is at (0, +2).

The convenience store is at (0, +1).

Therefore, instead of the pool being south of the convenience store, the pool is actually located to the north of the convenience store if you consider "north" to mean increasing y-coordinates. The convenience store, being at (0, +1), is southward relative to the school but not to the pool. The pool, being at the origin point (0,0), is south of both the school and the convenience store in terms of their y-coordinates. Thus, the third statement, "The pool is south of the convenience store," is false according to the given coordinates.

---

It gets coordinates and everything right, but still somehow messes up North and South.

Re:Gemini Ultra Released by Alifatisk on Friday February 09, 2024 @06:07AM (#39313636)
You know you can share conversations right?
Re:Gemini Ultra Released by aaronharnly on Thursday February 08, 2024 @09:09AM (#39302066)
Nice one! Gemini got it wrong in my (one) test:

https://g.co/gemini/share/4bae8ca3dd0a

Re:Gemini Ultra Released by microtherion on Thursday February 08, 2024 @10:30AM (#39303054)
Gemini happened to get the yes/no answer right, but the reasoning it gave is completely wrong, so it should not get any credit for it.