Like the intellectual equivalent of a beast of burden - good at solving open problems by (apparently) intuiting connections and making inferences, probably with the appearance of creative operation and experimental learning, but lacking any genuine desire or personal agency.

IMO this is much more plausible as AGI than some kind of mythical general meta-intelligence that has a virtual soul.

I don't see why consciousness or agency are at all necessary for AGI - except as a science fiction trope.

Besides which, consciousness and agency are two different things, damnit! I've got a github repo in which I train agents by policy optimization. I have no illusions that my Pytorch models are conscious.

>good at solving open problems by (apparently) intuiting connections and making inferences, probably with the appearance of creative operation and experimental learning, but lacking any genuine desire or personal agency.

so, like a philosophical zombie? like something that appears to be conscious but isn't?

I mean, we could call it an "artificial general intelligence" or we could call it a "generalized optimizer & problem solver"

So basically if we can have a system that can solve any problem thrown at it, that's essentially the AGI.

A really efficient slave. Essentially it does not get creative and is ultimately predictable, even in failure modes. It would also have no subconscious drives to do unexpected things.

Capitalistic dream.

Asimov's Three Laws of Perfect Slavery (Forgive me Isaac!)

If such beings were conscious they would be saints.

One of my favorite AI jokes:

Q: What's AI?

A: When the machine wakes up and says, "What's in it for me?"

It would be fetish to actually make humanoid servitor robots. IMO.

without creativity... I don't think it would make for a very effective slave in a world that is constantly changing.

Is creativity possible without consciousness? I don't pretend to be an expert.

Probably possible, yes. Subconscious drives are a thing. You could have sentience without consciousness. (The difference is that consciousness can change or modify drives and recognize its own actions on the world - its own agency. It is related to sentience and the line between the two is thin and blurry.) I'm not sure it is possible to be conscious without having been at some point sentient.

The idea was that such an AI would never question its priorities or change them, not that it wouldn't be able to solve novel problems.

(Which is the definition of intelligence, duh. And shows how far we're from that. What we have is glorified pattern matching algorithms and some basic symbolic logic and clustering.)

That sounds more like a Communist dream. Capitalism doesn’t reward really efficient slaves.