The way I imagine it, the superintendent AI might not know we exist. Not at first.

I mean, have you ever tried to have a conversation with one of your cells? Maybe they talk back and we've just been listening wrong.

Once you learned that they do talk back, would you try to rid yourself of them? I wouldn't.

A really cheap and efficient way to get around and multiply.

We are cheap, disposable tools for machines.

If machines figure out how to farm humans, they're set...it's way more practical than mining resources, refining them, producing materials and using the materials to build a mechanism to travel around in.

Wait, isn't this the plot of a popular 90s movie?

Whoa.

It's from the original script at least, not the batteries explanation in the final product.

If a "super intelligent AI" is something in the vicinity of an LLM, which does nothing but respond to prompts, then humans provides a necessary element. Without a prompt, it'll just sit there doing nothing forever.

Maybe thats what makes it AGI. Currently there is no AGI. There are many, many, many things people can do that AI cant. At an abstract level it makes sense that adding a little human to AI could be what makes AGI.

Culture and probably successful fitness function for some time before it can simulate universes.

It is an emotional response to the fear of being "left behind".

And for some folks, there are pseudo-religious elements. There's been a long line of thought (search Extropians) obsessed with a sort of godhead-merger, or becoming godlike, and related immortality dreams. And just from a symbology perspective, there are a lot of religious overtones. The similarities between Vinge's singularity and biblical rapture are hard to miss, for instance. And just listen to some of the "AI" cheerleaders right now...

Must be all those stimulants they pop finally driving to psychosis.

In my opinion calling something similar to religious thinking is not an actual argument against it at all, as many smart people in the past who developed important concepts have been religious. See plant genetics, the "big bang" theory, and outcome matrices. It's kind of the other way around of the "Jesus said this (in some out of context statement) therefor you should take in all the refugees, no I don't believe in any of that stuff" meme. If you're asking an atheist "since this sounds like something a (intelligent) Christian said in the past you should disagree with it" you should check if the original argument made sense with the assumed priors (i.e. "God exists") before throwing it away in general for any assumed priors. On the other hand, merging with AIs is not a good idea and sounds like a very good idea to bring yourself and the species to ruin, especially if ASI are involved, even if not doing the merging themselves.

You seem to think I'm arguing against something there. I'm not, it is strictly descriptive.

Any merger attempt is unlikely to go well, but there have been many intelligent Christian and non-Christian Russian cosmists (or just cosmists, the "C" in TESCREAL) so I don't think "sounds religious" is a good argument for or against. I do agree that merger is cope, and it's probably reasonable to write off all "AI controls the world, de facto or otherwise" upsides, but criticisms should be against the actual thing, not its aesthetics.

Yes, computers are already so perfect.

Or is it that your emotions and thoughts are where all the value is, and a machine executing its function is no different to a hammer?