> I think one form of cognitive labor will remain: What do we actually want to do/build.

Sure, but you don't need very many people to do that relative to the billions of workers that currently exist.

If you wipe out 90% of all jobs instead of 100% the practical implications for society as a whole are roughly exactly the same.

The Great Depression was "only" 25% unemployment.

A good precedent is mechanized agriculture. When machines effectively replaced most humans in agriculture, unemployment did not rise to 90%.

> A good precedent is mechanized agriculture. When machines effectively replaced most humans in agriculture, unemployment did not rise to 90%.

I don't consider this a good precedent at all.

When this happened, humans went off and started doing what we now consider "knowledge work". They escaped massive unemployment by being smarter than the tractors. How do they escape AI job loss when AI becomes smarter than the average human? or smarter than the smartest human?

In prior displacements humans moved up the stack. There is no longer room for them at higher levels of the stack if AI beats them at cognitive labor.

Yes but what about all the horses replaced by machines?

Humans keep a few around for entertainment but that’s about it.

what a great way of putting it, saving this

> Yes but what about all the horses replaced by machines?

The point is that people, like horses, can do other things. Humans don't even have to be better than robots at anything. It's the principle of comparative advantage.

No amount of comparative advantage can help you when your economic values is less than the cost of keeping you alive.

> The point is that people, like horses, can do other things.

...like what? If we start with the assumption that AI can broadly replace humans in knowledge jobs, what industries are absorbing tens of millions of humans for labor?

I see this as a categorical difference. The difference between AI and agricultural mechanization is that AI is highly heterogeneous. During the mechanization of agriculture that you talk about (which also happened over generations), there were alternative industries to move to. What happens when AI is competing in the majority of industries? Are there going to be 10 million new plumbers?

And this is why we must ensure that the AIs get aligned towards humanity and crucially that they don't get aligned towards capital, which seems to be the current trajectory.

There's a better chance that a humanity-aligned AI would make sure that humans have something to do and keep us alive versus a capital-aligned one. At the same time, if AI would be the deathblow to capitalism, that'd be very, _very_ ironic.

Only if the AI still cares about our opinion.

People already routinely ask AI questions about what to build or do.

Of course, and then AI will give you generic canned things to build. It cannot (yet) read minds. I don't understand what you wanted to make clear to me with this.

Sorry i wasn't clearer. My point is that some percent or those decisions are already outsourced to ai. When it makes decisions while carrying out work, or even in the choices and ideas it presents to us, it is shaping and deciding what gets built. Algorithms on tiktok and YouTube already decide what we watch. Probably most human watch hours in a day are decided by algorithms already.