News for hackers. Stuff that matters.
Re:Levels of AGI: Operationalizing Progress on the Path to AGI by adampwells on Monday November 13, 2023 @06:55PM (#38256996)
I think they make a category error by putting ChatGPT etc in the General column. As far as I can tell we only have narrow definitions of 'intelligence' and ChatGPT falls into one of those. I don't know of a general agreement on what 'General Intelligence' is in people, so how can we categorise anything is AGI? Knowing a bit about how ChatGPT works I feel it is a lot more like a chess program than a human.
Re:Levels of AGI: Operationalizing Progress on the Path to AGI by TeMPOraL on Monday November 13, 2023 @07:13PM (#38257143)
ChatGPT is the most general system ever created and packaged. You can throw arbitrary problems at it and get half-decent solutions for most of them. It can summarize and expand text, translate both explicitly and internally[0], play games, plan, code, transcode, draw, cook, rhyme, solve riddles and challenges, do basic reasoning, and many, many other things. Whether one is leaning more towards "stochastic parrot" or more towards "sparks of AGI" - it's undeniable that it's a general system.

--

[0] - The whole "fine-tune LLM on a super-specific task but only in language X (which is not English), its performance for that task improves in languages other than X" part, indicating it's not just learning tokens, but the meanings behind them too.

Re:Levels of AGI: Operationalizing Progress on the Path to AGI by Barrin92 on Tuesday November 14, 2023 @12:23AM (#38259480)
>it's undeniable that it's a general system.

it's very deniable, as Yann Lecun correctly pointed out, it can't even walk up a set of stairs (https://twitter.com/ylecun/status/1721648856260050970).

It can't cook, it can talk about cooking. It wouldn't be able to get a pan out of a drawer. I know all we do these days is produce text tokens on the internet, but it is in fact in itself a domain specific task. If you can talk about opening a can of beans you're an LLM if you can do that and actually open the physical can we may be a little bit further towards general intelligence.

We don't even have a full self driving system, and definitely not one that can chat with you about the news.

Re:Levels of AGI: Operationalizing Progress on the Path to AGI by ben_w on Tuesday November 14, 2023 @04:35AM (#38261047)
If that was a valid criticism of its intelligence, Stephen Hawking would have spent most of life categorised as a vegetable.
Re:Levels of AGI: Operationalizing Progress on the Path to AGI by Barrin92 on Tuesday November 14, 2023 @07:29AM (#38262488)
Stephen Hawking lost the capacity to move because his ALS paralyzed him, not because his brain lacked the capacity to do so, come on this has to be the worst analogy of the year. Also no, driving systems are not LLMs. LLMs are large language models, no existing self driving system runs on a language model. And also, that's not what the word "tautology" means. "All bachelors are unmarried" is a tautology.
Re:Levels of AGI: Operationalizing Progress on the Path to AGI by og_kalu on Tuesday November 14, 2023 @09:56AM (#38263999)
Re:Levels of AGI: Operationalizing Progress on the Path to AGI by ben_w on Tuesday November 14, 2023 @11:19AM (#38265286)
Ah, you wrote unclearly, it sounded like you were asserting that no system was an LLM rather than no driving system.

So while your claim is still false, I will accept that it isn't tautologically so.

Likewise, I am demonstrating that the actual definition you're using here is poor due to the consequence of it ruling out Stephen Hawking, and that goal means that the reason why he couldn't do things is unimportant: you still ruled him out with your standard.

Transformer models are surprisingly capable in multiple domains, so although ChatGPT hasn't got relatively many examples of labeled motor control input/output sequences and corresponding feedback values, this was my first search result for "llm robot control": https://github.com/GT-RIPL/Awesome-LLM-Robotics (note several are mentioning specifically ChatGPT).

Re:Levels of AGI: Operationalizing Progress on the Path to AGI by og_kalu on Tuesday November 14, 2023 @09:58AM (#38264026)
There are millions of people who cannot do any of those things, never could and will never be able to.

At any rate, language models are in fact able to do those things.

https://tidybot.cs.princeton.edu/

https://deepmind.google/discover/blog/rt-2-new-model-transla...

https://wayve.ai/thinking/lingo-natural-language-autonomous-...

Re:Levels of AGI: Operationalizing Progress on the Path to AGI by og_kalu on Monday November 13, 2023 @07:02PM (#38257051)
That's not a category error. GPT is general. It is able to perform many tasks (Creative Writing, playing Chess, Poker and other games, language translation, Code, robot piloting etc
Re:Levels of AGI: Operationalizing Progress on the Path to AGI by seanhunter on Tuesday November 14, 2023 @12:30AM (#38259502)
But not make toast. That's a general task that very nearly any human (intelligent or otherwise) can do.

If we define "general" as tasks with text input and output we are severely restricting the domain.

Re:Levels of AGI: Operationalizing Progress on the Path to AGI by og_kalu on Tuesday November 14, 2023 @10:01AM (#38264059)
There are millions of humans who can't make toast and will never be able to.

At any rate, no it's not really limiting.

https://tidybot.cs.princeton.edu/

https://deepmind.google/discover/blog/rt-2-new-model-transla...

https://wayve.ai/thinking/lingo-natural-language-autonomous-...

Re:Levels of AGI: Operationalizing Progress on the Path to AGI by ben_w on Tuesday November 14, 2023 @01:01PM (#38266963)
Arguments like yours are why I regard "generality" (certainly in the context of AGI) as a continuum rather than a boolean. AlphaZero is more general than AlphaGo Zero, as the former can do three games and the latter only one. All LLMs are much more general than those game playing models, even if they aren't so wildly superhuman on any specific skill, and `gpt-4-vision-preview` is more general than any 3.5 model as 4 can take image inputs while the 3.5's can't.
Re:Levels of AGI: Operationalizing Progress on the Path to AGI by seanhunter on Tuesday November 14, 2023 @03:20PM (#38269124)
Yes. If you read "Computing Machinery and Intelligence" this idea of generality being a continuum is a point that Turing makes actually (albeit in different words). What constitutes generality of an AI is really going to be very sensitive to your metric and the assessment is going to vary a lot from observer to observer.