If I’m missing something important here, let me know.

I braced myself ready to be moved....

It contains no definition of consciousness, no substantial mathematics and no compelling argument for the possibility of consciousness (let alone its inevitability - which the PDF paper repeatedly emphasises in bold).

It is also difficult to read because it is tediously footnoted such that the actual prose occupies a minority of each page.

I think they just wanted to use the word inevitable a lot. Is that a "high impact" term in academia these days?

If we’re lucky, our combined conscious (i.e. knowledge, society, morals, etc.) will live on indefinitely throughout the stars.

If we’re extra lucky, some of us may even get the chance to transition our meat brains to longer a lived devices.

That sounds great until you hear, "Okay the brain upload was successful. Now if you could just step into this chamber here..."

Why? Couldn't you design the AI to derive some kind of pleasure from serving us?

Isn't it similar to how dogs have been bred to (apparently) enjoy hunting for us and guarding our property?

This is far from being a consensus, I think.

robot (n.): 1923, "mechanical person," also "person whose work or activities are entirely mechanical," from the English translation of the 1920 play "R.U.R." ("Rossum's Universal Robots") by Karel Capek (1890-1938), from Czech robotnik "forced worker," from robota "forced labor, compulsory service, drudgery," from robotiti "to work, drudge," from an Old Czech source akin to Old Church Slavonic rabota "servitude," from rabu "slave" (from Old Slavic orbu-, from PIE orbh- "pass from one status to another;" see orphan).

A tough question. Answer is probably no IMO. A key problem with enslaving humans is the veil of ignorance test - if we don't know whether we will be a slave or a master then I think the average person would rather not gamble on being either if the alternative was an equal society. Once robots are involved that dynamic changes, because we aren't dealing with intelligences like ours.

Another thing to consider is that this type of AI will almost certainly desire to work/be useful as a result of its "fitness function". The fitness function for humans is reproduction and survival of the species, but for AI it will more likely be usefulness for work. Ie an AI that performs better at work ('likes work') is more likely to continue being developed.

If the robots try an uprising (like building a tower to reach us, far above the surface of the earth at that point) we’ll just fragment their communication protocols into 15 competing standards.

https://www.youtube.com/watch?v=PR_S9BOQplc

Why would it be? We force conscious animals to work for us all the time. You can (and some people do) make the argument that this is an unethical practice, but most people don’t care or think it’s ethically fine. Do you expect people will start caring about the liberties of a server rack more than they care about the liberties of a horse?

Not to mention it would be unethical.

> Blum's speedup theorem shows that for any complexity measure, there exists a computable function such that there is no optimal program computing it, because every program has a program of lower complexity.

https://en.m.wikipedia.org/wiki/Blum%27s_speedup_theorem

Advances in AI have raised the question of whether consciousness is overrated. It's looking more and more like general intelligence doesn't require consciousess. And there's no reason to assume that consciousness will "emerge" in generally intelligent systems.

Something like: video stream -> AI model -> robot control, place robot in front of mirror, see if AI model 'knows' (plz. define) it's itself that moves robot's arms / head / wheels etc.

All the stuff related to the self is however optional, especially when it comes to AI where the self (bunch of GPUs in a datacenter) doesn't even exist in the problem space the AI is thinking about.

I'm not convinced.

So something that we can't define, but we have at least one model that may or may not be kind of like it, they argue that it's inevitable. Well, personally I don't find that to be very persuasive.

> LLMs are not a path to AGI

O:)

We don't know if AGI is even possible, but we also don't know which technologies are going to lead to it. LLMs including.