For me, "AGI" would come in with being able to reliably perform simple open-ended tasks successfully without needing any specialized aid or tooling. Not necessarily very well, just being capable of it in the first place.

For a specific example of what I mean, there's Vending-Bench - even very 'dumb' humans could reliably succeed on that test indefinitely, at least until they got terminally bored of it. Current LLMs, by contrast, are just fundamentally incapable of that, despite seeming very 'smart' if all you pay attention to is their eloquence.

If someone handed you an envelope containing a hidden question, and your life depended on a correct answer, would you rather pick a random person out of the phone book or an LLM to answer it?

On one hand, LLMs are often idiots. On the other hand, so are people.

I'd learn as much as I could about what the nature of the question would be beforehand and pay a human with a great track record of handing such questions.

That's not at all analogous to what I'm talking about. The comparison would be picking an LLM or a random person out of the phone book to, say, operate a vending machine... and we already know LLMs are unable to do that, given the results of Vending-Bench.

More than 10% of the global population is illiterate. Even in first world countries, numeracy rates are 75-80%. I think you overestimate how many people could pass the benchmark.

Edit - rereading, my comment sounds far too combative. I mean it only as an observation that AI is catching up quickly vs what we manage to teach humans generally. Soon, if not already, LLMs will be “better educated” than the average global citizen.

And yet, I would be completely confident that an average illiterate person could pass the Vending-Bench test indefinitely if you gave them interfaces that don't depend on the written word (phone calls, abacuses, piles of blocks, whatever), and that the "smartest" LLM in the world couldn't. It's not about level of education, beyond the bare minimum needed to have any kind of mental model of the world.

Where would you draw the line? Any ol' computer outperforms me in doing basic arithmetic.

This is a question of how we quantify intelligence, and there aren’t many great answers. Still, basic arithmetic is probably not the right guideline for intelligence. My guess has always been that it’ll lie somewhere in ability to think critically, which they still have not even attempted yet, because it doesn’t really work with LLMs as they’re structured today.

I'd suggest anything able to match a professional doing knowledge work. Original research from recognisably equivalent cognition, or equal abilities with a skilled practitioner of (eg) medicine.

This sets the bar high, though. I think there's something to the idea of being able to pass for human in the workplace though. That's the real, consequential outcome here: AGI genuinely replacing humans, without need for supervision. That's what will have consequences. At the moment we aren't there (pre-first-line-support doesn't count).

At the very least, it needs to be able to collate training data, design, code, train, fine tune and "RLHF" a foundational model from scratch, on its own, and have it show improvements over the current SOTA models before we can even begin to have the conversation about whether we're approaching what could be AGI at some point in the future.

I cannot do all that. Am I not generally intelligent?

You could if you were trained to do so. LLMs cannot, even if they are.

Are LLMs trained to do all that? How do you know they cannot?

> Are LLMs trained to do all that

Yes, they are trained to do all that more or less as much as anything else they are trained to do.

> How do you know they cannot?

The same way I know they can't run a fortune 500 company. It's clearly apparent.

Last time I checked they were trained to predict the next word. Why would you expect them to solve research problems?

> Why would you expect them to solve research problems?

I don't.

Great, then we established that I (generally intelligent human), and LLMs, both can't perform the task in question without specific training. We still don't know if given that specific training we (a specific human or a specific LLM) would be able to perform it.

That would be human; I've always understood the General to mean 'as if it's any human', i.e. perhaps not absolute mastery, but trained expertise in any domain.

no it doesn't has to it just has to be "general"

as in it can learn by itself to solve any kind of generic task it can practically interface it (at lest which isn't way to complicated).

to some degree LLMs can do so theoretically but

- learning (i.e. training them) is way to slow and costly

- domain adoption (later learning) often has a ton of unintended side effects (like forgetting a bunch of important previously learned things)

- it can't really learn by itself in a interactive manner

- "learning" by e.g. retrieving data from knowledge data base and including it into answers (e.g. RAG) isn't really learning but just information retrieval, also it has issues with context windows and planing

I could imagine OpenAI putting together multiple LLMs + RAG + planing systems etc. to create something which technically could be named AGI but which isn't really the break through people associate with AGI in the not too distant future.