News for hackers. Stuff that matters.
Re:We could stumble into AI catastrophe by Animats on Friday January 13, 2023 @01:25PM (#34371795)
An important point is that AI does not need to be sentient or conscious to do a lot of damage. An AI that is simply very good at optimizing a particular task can do a significant amount of damage inadvertently, if it turns out that task in the limit is bad for humans.

We have that now. It's called a corporation. See yesterday's article about Exxon.

Re:We could stumble into AI catastrophe by FeepingCreature on Friday January 13, 2023 @01:25PM (#34371802)
If corporations could increase their operational intelligence, they would be an existential threat to life on earth. (More so, anyways.)
Re:We could stumble into AI catastrophe by actually_a_dog on Friday January 13, 2023 @01:26PM (#34371819)
See also "paperclip maximizer." Then imagine what happens when you have an entire global economy driven by these types of inhuman, amoral agents.

https://www.lesswrong.com/tag/paperclip-maximizer

Re:We could stumble into AI catastrophe by qwertyforce on Friday January 13, 2023 @02:16PM (#34372509)
I've also read somewhere about AI feedback loops:

ai generates content <--money--> ai pushes ads to promote its content.

It's somewhat already happening: recommendation algorithms choose what content we consume, analytics make decisions from this data and make more content that recommendation algorithms like. AI HFT bots can directly influence economy and so on

Re:We could stumble into AI catastrophe by Oxidation on Friday January 13, 2023 @02:18PM (#34372527)
I would say that "billionaire wealth maximization" agents are already giving us a glimpse.

Or rather, they're giving the rest of us a glimpse of what life has been like for those at the "bottom of the pile" for a long time: a world organised around goals essentially completely unrelated to those people's existence.

Re:We could stumble into AI catastrophe by gala8y on Friday January 13, 2023 @01:56PM (#34372228)
Which one?
Re:We could stumble into AI catastrophe by hn_throwaway_99 on Friday January 13, 2023 @02:12PM (#34372454)
Re:We could stumble into AI catastrophe by JumpCrisscross on Friday January 13, 2023 @01:58PM (#34372273)
> called a corporation

That’s banal, self-interested human evil. Corporations and governments alike have perpetuated mass crimes and cover-ups.

Re:We could stumble into AI catastrophe by Animats on Friday January 13, 2023 @02:25PM (#34372613)
Corporations are designed to do so. So spake Milton Friedman.[1] That was when "greed is good, greed works" became respectable.

[1] https://www.nytimes.com/1970/09/13/archives/a-friedman-doctr...

Re:We could stumble into AI catastrophe by JumpCrisscross on Saturday January 14, 2023 @01:17AM (#34377784)
This applies to all human institutions. Friedman studied private enterprise, so his insights are thereby scoped. Ecologists recognise inter-species competition; that doesn’t make it the exclusive domain of life.
Re:We could stumble into AI catastrophe by WhitneyLand on Friday January 13, 2023 @02:20PM (#34372562)
>We have that now. It's called a corporation.

I think there’s a good argument to say no, we don’t have that now, and these are some ways it could be very different.

Video: Why not just think of AGI as a corporation?

https://youtu.be/L5pUA3LsEawP

Re:We could stumble into AI catastrophe by zmmmmm on Friday January 13, 2023 @03:51PM (#34373716)
At least corporations are currently limited by requiring humans who tend to have at least some limits to their actions in terms of moral, ethical considerations and can be physically constrained by imprisoning them etc.

An AI could be completely untethered from that and set new thresholds for evil corporate behaviour.

Re:We could stumble into AI catastrophe by taberiand on Friday January 13, 2023 @04:10PM (#34373915)
And yet even with current constraints, corporations are capable of destroing the world through climate change and other ecological disasters. Humans are also limited by their personal greed, hubris, selfishness, myopic views and general incompetence.

An AI could be completely untethered from that and set new thresholds for ethical corporate behaviour that operates on fundamentally ethical foundations of caring for the environment and humanity

Re:We could stumble into AI catastrophe by moorg on Friday January 13, 2023 @04:34PM (#34374203)
Good observation. It reminds of Etsy giving up its status as a B Corp[1], presumably due to pressure from shareholders. Once the ship is set in motion, it can't divert its course even if it wanted to.

1: https://www.etsy.com/news/business-as-a-force-for-good-defin...