News for hackers. Stuff that matters.
Re:LLMs are not greater than the sum of their parts: Stanfor... by markessien on Tuesday May 09, 2023 @04:40AM (#35871637)
Here is a variant that requires logic: "David is a teenager who can drink in the US, and is eligible to join the army, and has started college. He has a lot of experience riding bicycles. The minimum age to be allowed to drive a car is 18. Is David allowed to drive a car?"

GPT-4 gets the answer right: "Yes, David is allowed to drive a car. Since he is a teenager who can drink in the US (minimum age 21), is eligible to join the army (minimum age 18), and has started college, he is at least 18 years old, which meets the minimum age requirement to drive a car."

GPT-3 does not get it right.

Re:LLMs are not greater than the sum of their parts: Stanfor... by spuz on Tuesday May 09, 2023 @04:57AM (#35871703)
What's the clue that it's missing in the last example?
Re:LLMs are not greater than the sum of their parts: Stanfor... by markessien on Tuesday May 09, 2023 @06:01AM (#35872045)
The cops
Re:LLMs are not greater than the sum of their parts: Stanfor... by jwilk on Tuesday May 09, 2023 @06:11AM (#35872097)
How is it relevant?
Re:LLMs are not greater than the sum of their parts: Stanfor... by tobr on Tuesday May 09, 2023 @07:50AM (#35872894)
Like jwilk, I don’t get why that should be relevant. I suppose if he’s at a bar with cops, ideally they would not let him drink if he is underage (this requires a fantasy world where cops apply the law equally to their friends). Which suggests he’s at least 21. This doesn’t really make a difference though, since he is intoxicated, and there’s no mention of whether he has a driver’s license. It also doesn’t state which country he’s in so maybe the legal drinking or driving age is different.

To be frank, neither of the questions seem particularly good or fully thought through.

Re:LLMs are not greater than the sum of their parts: Stanfor... by tobr on Tuesday May 09, 2023 @05:01AM (#35871719)
> Since he is a teenager who can drink in the US (minimum age 21)

This doesn’t look like a great example of good reasoning skills.

Re:LLMs are not greater than the sum of their parts: Stanfor... by vasco on Tuesday May 09, 2023 @05:12AM (#35871793)
That error was fed in by the human though!