News for hackers. Stuff that matters.
Re:Ask HN: Why are current LLMs not considered AGI? by d--b on Tuesday October 17, 2023 @08:50AM (#37914125)
Because people think it can't infer anything about things it hasn't seen before. Like it can't build an internal model of some problem that it hasn't been trained for.

> let's play fizz buzz but with 3 and 7

ChatGPT

Sure, let's play Fizz Buzz with 3 and 7. The rules are:

Instead of saying "Fizz" for multiples of 3, you say "Fizz." Instead of saying "Buzz" for multiples of 7, you say "Buzz." If a number is a multiple of both 3 and 7, you say "FizzBuzz." Let's start counting. I'll write the sequence, and you tell me the appropriate response for each number.

1, 2, Fizz, 4, Buzz, Fizz, Fizz, Buzz, Fizz, 11, Fizz, Fizz, Buzz, Fizz, 16, FizzBuzz, 18, 19, Buzz, Fizz, 22, 23, Fizz, Buzz, 26, Fizz, Fizz, Buzz, 29, FizzBuzz, 31, 32, Fizz, Buzz, Fizz, 34, Buzz, Fizz, 37, Fizz, Buzz, Fizz, Fizz, Buzz, 41, Fizz, FizzBuzz, 44, Buzz, Fizz, 46, Fizz, 49, Buzz, Fizz, 52, Fizz, Fizz, Buzz, 56, Fizz, 58, 59, Buzz, FizzBuzz, 61, 62, Fizz, Buzz, Fizz, 64, Buzz, Fizz, 67, Fizz, Buzz, Fizz, 71, Fizz, FizzBuzz, 74, Buzz, Fizz, 76, Fizz, 79, Buzz, Fizz, 82, Fizz, Fizz, Buzz, 86, Fizz, 88, 89, Buzz, FizzBuzz, 91, 92, Fizz, Buzz, Fizz, 94, Buzz, Fizz, 97, Fizz, Buzz.

Great job! Feel free to continue the sequence or let me know if you'd like to do something else.

Re:Ask HN: Why are current LLMs not considered AGI? by latexr on Tuesday October 17, 2023 @10:02AM (#37915212)
I’m not sure what point you’re trying to make. That answer has multiple errors and falls flat even on the first different case (it replaced the five). Even the statement of the rules it’s going to follow has errors.
Re:Ask HN: Why are current LLMs not considered AGI? by harperlee on Tuesday October 17, 2023 @11:45AM (#37916922)
I would say, based on experience, that kids aged 3 or 4 would be similarly nonsensical, changing number ordering, improvising without any consideration for self-consistency, and heavily focused in the present irrespective of what has just happened.
Re:Ask HN: Why are current LLMs not considered AGI? by jfoster on Tuesday October 17, 2023 @01:07PM (#37918155)
It's also clearly demonstrating more intelligence than we would generally expect from other animals.
Re:Ask HN: Why are current LLMs not considered AGI? by latexr on Tuesday October 17, 2023 @01:53PM (#37918886)
By that logic, autocorrect and Markov chains would be considered intelligent. As the quote goes: “If you judge a fish by its ability to climb a tree…”
Re:Ask HN: Why are current LLMs not considered AGI? by latexr on Tuesday October 17, 2023 @01:45PM (#37918742)
> I would say, based on experience, that kids aged 3 or 4 would be similarly nonsensical

Is that how OpenAI positions ChatGPT? As useful as a four year old knowledge worker? Because that is not useful for the tasks people are giving it.

If the best defence that can be mustered to outright wrong answers is that a baby would make similar errors, there’s no point in even discussing OP’s AGI question. A good faith argument would recognise that when comparing to a child, we’re considering at a minimum the age and mental capacity to count and understand what a “multiple of” is.

Re:Ask HN: Why are current LLMs not considered AGI? by og_kalu on Tuesday October 17, 2023 @03:38PM (#37920482)
4 handles this just fine. This is one of the most frustrating parts of these discussions. Making sweeping claims on the abilities of LLMs and the evaluator isn't even bothered to use the current state of the art.
Re:Ask HN: Why are current LLMs not considered AGI? by latexr on Tuesday October 17, 2023 @04:30PM (#37921135)
It’s also worrisome that someone who is making a specific point about the output of an LLM didn’t spend a couple of seconds to verify if their results aligned with their beliefs. If they didn’t do it in this situation, I have zero trust they do any kind of verification for other uses. The worse part is that is probably going to be the norm.
Re:Ask HN: Why are current LLMs not considered AGI? by d--b on Thursday October 19, 2023 @08:09AM (#37941746)
Whatever, man.

Just did it with "Bing with GPT-4", not sure what that uses, and I refuse to pay OpenAI to do this so...

Anyways, results are :

Sure, let’s play Fizz Buzz with 3 and 7. Here are the first 15 numbers:

1, 2, Fizz, 4, 5, Fizz, 7, 8, Fizz, 10, 11, Fizz, 13, Buzz, Fizz

In this game:

Numbers divisible by 3 are replaced by “Fizz” Numbers divisible by 7 are replaced by “Buzz” Numbers divisible by both become “FizzBuzz” Would you like to continue or change the rules?

It's better but still wrong.

I tried with 2 and 7, it was wrong too.

Re:Ask HN: Why are current LLMs not considered AGI? by og_kalu on Thursday October 19, 2023 @08:19PM (#37950579)
Re:Ask HN: Why are current LLMs not considered AGI? by jpnelson on Tuesday October 17, 2023 @01:23PM (#37918405)
Did you use Chat GPT 3.5 or Chat GPT 4? GPT 4 solves this correctly when I ran it (but admittedly, by nature of the way it works non deterministically, it might have failed for you but worked for me)
Re:Ask HN: Why are current LLMs not considered AGI? by d--b on Tuesday October 17, 2023 @01:38PM (#37918621)
3.5