Everything old is new again.

It's going to be some damn thing when everyone in the South takes your idea and they build up their civilization around free robotic labor, and then when we hit AI sentience and start ensuring the civil rights of robots the entire South has to be burned to the ground again to free the sentient robots from their owners.

You don't have to make machines sentient, you know. Some people seem to think the goal of AI and robotics should be to make more humans, but we already have plenty of humans. We should make machines that do our work more efficiently so we don't have to do it anymore.

We can still do the fun stuff of course, like writing and art, and get famous and all that.

It used to be a stupid question to me since how could rules have consciousness.

But most modern AI is learnt from data, not programmed, how will we know if our AI in conscious?

We're not there yet and may never be, but since we do not know what consciousness is or what leads to it, we may stumble into it...

I agree, I'm just not sure we'll be able to convince everyone with the capability of making sentient robots that they shouldn't make sentient robots before they make sentient robots.

Human labor has solved a lot of problems. The obvious shortcut to solving lots of problems is mimicking human labor. Given the opportunity to take a shortcut to solve a problem (especially at a profit) some manager in some business will do it.

Zero people have the capability to make sentient robots at the moment.

And zero people can even describe how a sentient robot could be constructed, even with theoretical future hardware. "We'll have as many flops as some models say a brain has, and programmers are creative" is about as far as anyone can get, but I don't think that qualifies as a plan.

So, it's a fun thing to think about, but like all sci-fi it probably isn't the best basis for societal planning.

The question is whether a silicon brain can acquire the capabilities of a human brain without picking up sentience along the way.

In order to perform at or above human level (in a general fashion), the robot will need a firm, comprehensive understanding of the world. Won't this unavoidably involve an understanding of the self as well?

What self is there to understand? Does a computer chip know what it's processing?

The awareness of all sensory or data input, internal processes, as well as the system's dynamic conceptual models and their change over time. This includes an awareness of the system's conceptual model of itself.

I don't think any chips today have awareness. But it seems to me that you couldn't have general, adaptable intelligence without it.

But the land is all owned already, and not by the people who's jobs are being replaced? And so far automation in farming is focused on large-scale operation, with much less good solutions for family -scale.

Economies of scale have been the bread and butter of progress in the last 100 years, but we're entering the age of customization now. Adaptive technologies can work on small scale as well as on large scale. And large scale can still be achieved by coops that provide services for many people at once. People do have capital and resources, it's just spread and divided. What we need is to organize.

I give self-replicating factories a higher probability than AGI. We'll probably have a 100% automated stack that can replicate and repair itself, as well as produce everything we need, all based on locally sourced and plentiful materials, before we have human level AI. It will be a manufacturing singularity. We should bootstrap automation in itself, like we bootstrap compilers. Object production would become similar to compilation, from source code to material object. We'd have optimized material compilers.

When we have a replicator, anyone can have one. Industry would become like agriculture, if you have the initial seeds, you can plant and then you bootstrap your crop.

If you think about self replication, on the whole, industry is a self replicator right now. But we need to make it fully automated, reduce its minimal footprint and make it less reliant on rare materials. There is a need for sustainable, smart materials and open source automation and AI in order to save the population from disenfranchisement. Capital and the means of production should belong to all of us, not just the 1%.

Even if everyone has their own replicator, the competition shifts to who controls raw materials, and who controls energy.

Fresh water, for example, is a scarce (and becoming scarcer) resource, and it's impossible to farm or live without it.

Yes, although I'm not convinced that fresh water is going to be such a scarce resource in the future. There are a few "outs" on the horizon—besides changing our diet, the possibilities for desalinization, or things like lab-grown meats, may change our relationship with water. Just as long as population growth slows enough…

Agree with the comment about ownership. As technology has improved, more and more power has been going to whoever owns the capital.

Fresh water is not a problem given unlimited energy since desalination works well.

The problem is that energy is expensive, and fresh water shortages will be worst in countries that are poor and can't afford these plants.

I agree. That's why capitalists owning automation should pay UBI - because they get to use raw materials.

Whisky is for Drinking. Water is for Fighting. -Mark Twain