News for hackers. Stuff that matters.
Re:Llama 3 8B is almost as good as Wizard 2 8x22B by joelthelion on Friday April 19, 2024 @07:24AM (#40085432)
Llama 3 70B seems to handle it just fine:

A classic lateral thinking puzzle!

Let's break it down:

* Jack has 15 sisters. * Each sister has 3 brothers.

Now, here's the key insight: if each sister has 3 brothers, that means each sister is counting Jack as one of their 3 brothers!

So, how many brothers does Jack have? Only 2 The other 2 brothers that each sister is counting are actually Jack himself.

Answer: Jack has 2 brothers.

Re:Llama 3 8B is almost as good as Wizard 2 8x22B by MyFirstSass on Friday April 19, 2024 @08:13AM (#40085803)
Wow, that's extremely impressive in light of GPT being 1800b so a model 25 times smaller is beating it logic wise!

I don't have enough ram to run 70b but i'm very curious about performance on other logic puzzles!

Re:Llama 3 8B is almost as good as Wizard 2 8x22B by MyFirstSass on Friday April 19, 2024 @11:42AM (#40088219)
I just tried variations and weirdly it fails many of them like:

"bob has 7 sisters, each sister has 4 brothers. How many brothers does bob have?"

-----------------

Llama 70b instruct (wrong)

A classic lateral thinking puzzle!

At first glance, it seems like a straightforward math problem, but wait...

If each of Bob's 7 sisters has 4 brothers, that would mean each sister has Bob and 3 other brothers, right? But that's not possible, because Bob is the only brother!

So, the correct answer is: Bob has no brothers. He has 7 sisters, and each of them has him as their only brother.

Did I get it right?

-----------------

GPT-4 (correct)

To solve this, let's consider the family structure:

Bob has 7 sisters. Each of these sisters has 4 brothers. Since all the sisters have the same number of brothers, and these brothers include Bob, this implies Bob has 3 other brothers. Therefore, including Bob, there are 4 brothers in total. Hence, Bob has 3 brothers.