Everyone’s deeply up their own asses about AGI, existential risks, this will solve climate change, bias, etc etc.

Nobody seems to discuss the risk that millions of white-collar jobs with little clout (paralegals, admin clerks, logistics coordinators, some portion of software developers, etc) are eliminated.

I think the largest risk in the next 10-20 years is that the tech gets good enough to shed jobs without replacing them (certainly not at a 1:1 ratio), furthering inequality, social division, etc. in the US at least, it seems that any broad policy proposal to address this is DOA.

Economists have discussed it, and just like with previous automation, there'll always be new jobs as human wants are unlimited (and by the time AI are able to do every job humans can do, they won't be willing to do it for free).

Past performance is not indicative of future results.

> human wants are unlimited

Physical resources are limited, and producing work requires the allocation of capital. If people want to work but cannot access the basic tools and resources to work then they'll languish in unemployment. Society doesn't just default to a productive society with high employment; these conditions need to be nurtured. Given the current state of economic inequality we're trending away from conditions that will lead to full employment if AI wipes out large swathes of jobs.

Based on what evidence? Do you think better ML will create no new opportunity? What does economic inequality have to do with whether new companies or services are built using new ML tools?

The automation we've been working on for decades is aimed—often explicitly—at eliminating the need for human beings to do certain kinds of jobs. Any philosophy or policy that does not then say, "...and then we will use the extra productivity to support those people in perpetuity" is morally bankrupt as well as failing to recognize social and economic realities we've known for hundreds of years.

It's significant to note that these biases don't have to be overt political biases; it can be a predisposition to use virgin materials over recycled, a tendency to generate JavaScript over Julia, etc cetera. Our available datesets are images of us and thus aren't predisposed to surpass our limitations.

Given that AI models are bound to the same biases that society struggles with, which ones do we encode in the models and which ones do we mitigate? Do we try to acknowledge and mitigate those limitations or say "This is fine" and ship our current organizational and political limitations?

If we want to stop fighting the same damn wars over and over again then our machine models have to be designed to account for our biases and limitations. Do you have suggestions for alternate ways to improve our models aside from explicit error correction?

> If by default an LLM refers to engineers as "he" more than "she", it's because engineers are more likely to be male than female.

That's a great point to raise, thank you for mentioning it. This really is the crux - machine models crystallize _current_ bias and amplify it. Most programmers in the early days of computers were women, but were displaced over time. Should the training data reflect the bias of the dominance of women in the early days, the dominance of men now, or some other combination?

> Most people would prefer an AI whose model of reality matches actual reality

Training data is not reality; training data is training data. It will fundamentally lag behind reality, and it will have a bias towards past conditions and precedents.

You do realise, if you command an LLM to talk like a caveman it doesn't invent a time machine, travel back in time and research how cavemen actually talked; it just produces the "me hit with rock" that redditors imagine a caveman would have talked like.

I’m guessing people will realize at some point AGI can’t undo the great paradoxes of society like resource distribution and value conflict. And we will slowly learn that a lot of what keeps us where we are isn’t a lack of super intelligence. It’s that the search and map pattern for human survival in the universe is really really messy and the conflict is inherently a part of it to guard against over optimization paths.