News for hackers. Stuff that matters.
Re:We could stumble into AI catastrophe by dwohnitmok on Friday January 13, 2023 @01:16PM (#34371658)
An important point is that AI does not need to be sentient or conscious to do a lot of damage. An AI that is simply very good at optimizing a particular task can do a significant amount of damage inadvertently, if it turns out that task in the limit is bad for humans.

This is especially the case because it seems like private individuals and companies are racing to hook up AIs to all sorts of real world systems, both digital and physical, to give them as much tangible impact as possible.

With the rapid rise of AI capabilities, where groundbreaking advances are being measured in weeks rather than years, I strongly urge developers to think harder about AI safety. AI safety shouldn't be a dirty word that has quotes surrounding it to delegitimize it as a farfetched concern. Benchmarks of things that we thought AIs couldn't do or were years if not decades out are falling by the day.

Re:We could stumble into AI catastrophe by rcarr on Friday January 13, 2023 @01:40PM (#34372014)
> Benchmarks of things that we thought AIs couldn't do or were years if not decades out are falling by the day.

I wonder, with crypto crashing and potentially becoming worthless, if all those mining cards will get sold on cheaply and repurposed for AI and we see another explosion in AI capability.

Re:We could stumble into AI catastrophe by Animats on Friday January 13, 2023 @01:25PM (#34371795)
An important point is that AI does not need to be sentient or conscious to do a lot of damage. An AI that is simply very good at optimizing a particular task can do a significant amount of damage inadvertently, if it turns out that task in the limit is bad for humans.

We have that now. It's called a corporation. See yesterday's article about Exxon.

Re:We could stumble into AI catastrophe by FeepingCreature on Friday January 13, 2023 @01:25PM (#34371802)
If corporations could increase their operational intelligence, they would be an existential threat to life on earth. (More so, anyways.)
Re:We could stumble into AI catastrophe by actually_a_dog on Friday January 13, 2023 @01:26PM (#34371819)
See also "paperclip maximizer." Then imagine what happens when you have an entire global economy driven by these types of inhuman, amoral agents.

https://www.lesswrong.com/tag/paperclip-maximizer

Re:We could stumble into AI catastrophe by qwertyforce on Friday January 13, 2023 @02:16PM (#34372509)
I've also read somewhere about AI feedback loops:

ai generates content <--money--> ai pushes ads to promote its content.

It's somewhat already happening: recommendation algorithms choose what content we consume, analytics make decisions from this data and make more content that recommendation algorithms like. AI HFT bots can directly influence economy and so on

Re:We could stumble into AI catastrophe by Oxidation on Friday January 13, 2023 @02:18PM (#34372527)
I would say that "billionaire wealth maximization" agents are already giving us a glimpse.

Or rather, they're giving the rest of us a glimpse of what life has been like for those at the "bottom of the pile" for a long time: a world organised around goals essentially completely unrelated to those people's existence.

Re:We could stumble into AI catastrophe by gala8y on Friday January 13, 2023 @01:56PM (#34372228)
Which one?
Re:We could stumble into AI catastrophe by hn_throwaway_99 on Friday January 13, 2023 @02:12PM (#34372454)
Re:We could stumble into AI catastrophe by JumpCrisscross on Friday January 13, 2023 @01:58PM (#34372273)
> called a corporation

That’s banal, self-interested human evil. Corporations and governments alike have perpetuated mass crimes and cover-ups.

Re:We could stumble into AI catastrophe by Animats on Friday January 13, 2023 @02:25PM (#34372613)
Corporations are designed to do so. So spake Milton Friedman.[1] That was when "greed is good, greed works" became respectable.

[1] https://www.nytimes.com/1970/09/13/archives/a-friedman-doctr...

Re:We could stumble into AI catastrophe by JumpCrisscross on Saturday January 14, 2023 @01:17AM (#34377784)
This applies to all human institutions. Friedman studied private enterprise, so his insights are thereby scoped. Ecologists recognise inter-species competition; that doesn’t make it the exclusive domain of life.
Re:We could stumble into AI catastrophe by WhitneyLand on Friday January 13, 2023 @02:20PM (#34372562)
>We have that now. It's called a corporation.

I think there’s a good argument to say no, we don’t have that now, and these are some ways it could be very different.

Video: Why not just think of AGI as a corporation?

https://youtu.be/L5pUA3LsEawP

Re:We could stumble into AI catastrophe by zmmmmm on Friday January 13, 2023 @03:51PM (#34373716)
At least corporations are currently limited by requiring humans who tend to have at least some limits to their actions in terms of moral, ethical considerations and can be physically constrained by imprisoning them etc.

An AI could be completely untethered from that and set new thresholds for evil corporate behaviour.

Re:We could stumble into AI catastrophe by taberiand on Friday January 13, 2023 @04:10PM (#34373915)
And yet even with current constraints, corporations are capable of destroing the world through climate change and other ecological disasters. Humans are also limited by their personal greed, hubris, selfishness, myopic views and general incompetence.

An AI could be completely untethered from that and set new thresholds for ethical corporate behaviour that operates on fundamentally ethical foundations of caring for the environment and humanity

Re:We could stumble into AI catastrophe by moorg on Friday January 13, 2023 @04:34PM (#34374203)
Good observation. It reminds of Etsy giving up its status as a B Corp[1], presumably due to pressure from shareholders. Once the ship is set in motion, it can't divert its course even if it wanted to.

1: https://www.etsy.com/news/business-as-a-force-for-good-defin...

Re:We could stumble into AI catastrophe by ajuc on Friday January 13, 2023 @01:24PM (#34371781)
Corporations are AIs implemented on brains. And they are profit-maximizers. And they routinely do A LOT of damage. They even have legal rights :)

We're just digitizing that.

Re:We could stumble into AI catastrophe by quadcore on Friday January 13, 2023 @02:10PM (#34372428)
An important point is that AI does not need to be sentient or conscious to do a lot of damage.

My pet theory is that consciousness could be (not saying I know for sure this is the case but it could as well be) an effect of "survival instinct". That is: "i'll take action to not die".

Now this is very easy to learn from us as we do this all day long so to speak. Most of the way people react on the internet is tinted by the fear of death and survival for example.

What Im saying is that what we intuitively think as very complex could actually be very simple. As simple as some task optimization. Knowing the only thing that could unplug an AI is us, that'd lead to a hell of a fight. And maybe our complete termination.

Re:We could stumble into AI catastrophe by hooande on Friday January 13, 2023 @05:55PM (#34375051)
> "i'll take action to not die"

This thought is entirely the result of evolution. Beings that didn't have a survival instinct didn't live to reproduce, so they don't have descendants that we can observe.

There's no guarantee that a survival instinct is a necessary or even desirable element of conscious thought. You're just describing survivorship bias in its most literal form. cue the picture of the shot up airplane

Re:We could stumble into AI catastrophe by chucklenorris on Friday January 13, 2023 @06:44PM (#34375491)
Yeah, i find the connection between consciousness and intelligence tenuous at best. Our survival imperative, thermodynamic and game theoretical constraints give rise to it and i'd argue that every living organism has a form of consciousnes, from very primitive to, well, our level. So yeah, I believe we'll have far more intelligent ais in the future that would otherwise be inert.

Ofcourse, in our foolishness, we could create an ai with the goal of survival and replication at the expense of everything else. Or self optimisation could inadvertedly do so

Re:We could stumble into AI catastrophe by grantcas on Saturday January 14, 2023 @01:46PM (#34382786)
It's becoming clear that with all the brain and consciousness theories out there, the proof will be in the pudding. By this I mean, can any particular theory be used to create a human adult level conscious machine. My bet is on the late Gerald Edelman's Extended Theory of Neuronal Group Selection. The lead group in robotics based on this theory is the Neurorobotics Lab at UC at Irvine. Dr. Edelman distinguished between primary consciousness, which came first in evolution, and that humans share with other conscious animals, and higher order consciousness, which came to only humans with the acquisition of language. A machine with primary consciousness will probably have to come first.

What I find special about the TNGS is the Darwin series of automata created at the Neurosciences Institute by Dr. Edelman and his colleagues in the 1990's and 2000's. These machines perform in the real world, not in a restricted simulated world, and display convincing physical behavior indicative of higher psychological functions necessary for consciousness, such as perceptual categorization, memory, and learning. They are based on realistic models of the parts of the biological brain that the theory claims subserve these functions. The extended TNGS allows for the emergence of consciousness based only on further evolutionary development of the brain areas responsible for these functions, in a parsimonious way. No other research I've encountered is anywhere near as convincing.

I post because on almost every video and article about the brain and consciousness that I encounter, the attitude seems to be that we still know next to nothing about how the brain and consciousness work; that there's lots of data but no unifying theory. I believe the extended TNGS is that theory. My motivation is to keep that theory in front of the public. And obviously, I consider it the route to a truly conscious machine, primary and higher-order.

My advice to people who want to create a conscious machine is to seriously ground themselves in the extended TNGS and the Darwin automata first, and proceed from there, by applying to Jeff Krichmar's lab at UC Irvine, possibly. Dr. Edelman's roadmap to a conscious machine is at https://arxiv.org/abs/2105.10461

Re:We could stumble into AI catastrophe by yamtaddle on Friday January 13, 2023 @04:02PM (#34373838)
> An important point is that AI does not need to be sentient or conscious to do a lot of damage.

It's not 100% clear that consciousness is necessary even for things like creative problem solving or goal-setting (i.e. choosing which problems to solve). That is, advanced intelligences that can do all we can and more—and could even "decide", in the worst case, that they'd be better off without humans—may not need consciousness as a precondition for that.

Cf. Peter Watts' Blindsight

http://www.rifters.com/real/Blindsight.htm

Re:We could stumble into AI catastrophe by grantcas on Friday January 13, 2023 @05:44PM (#34374949)
It's becoming clear that with all the brain and consciousness theories out there, the proof will be in the pudding. By this I mean, can any particular theory be used to create a human adult level conscious machine. My bet is on the late Gerald Edelman's Extended Theory of Neuronal Group Selection. The lead group in robotics based on this theory is the Neurorobotics Lab at UC at Irvine. Dr. Edelman distinguished between primary consciousness, which came first in evolution, and that humans share with other conscious animals, and higher order consciousness, which came to only humans with the acquisition of language. A machine with primary consciousness will probably have to come first.

What I find special about the TNGS is the Darwin series of automata created at the Neurosciences Institute by Dr. Edelman and his colleagues in the 1990's and 2000's. These machines perform in the real world, not in a restricted simulated world, and display convincing physical behavior indicative of higher psychological functions necessary for consciousness, such as perceptual categorization, memory, and learning. They are based on realistic models of the parts of the biological brain that the theory claims subserve these functions. The extended TNGS allows for the emergence of consciousness based only on further evolutionary development of the brain areas responsible for these functions, in a parsimonious way. No other research I've encountered is anywhere near as convincing.

I post because on almost every video and article about the brain and consciousness that I encounter, the attitude seems to be that we still know next to nothing about how the brain and consciousness work; that there's lots of data but no unifying theory. I believe the extended TNGS is that theory. My motivation is to keep that theory in front of the public. And obviously, I consider it the route to a truly conscious machine, primary and higher-order.

My advice to people who want to create a conscious machine is to seriously ground themselves in the extended TNGS and the Darwin automata first, and proceed from there, by applying to Jeff Krichmar's lab at UC Irvine, possibly. Dr. Edelman's roadmap to a conscious machine is at https://arxiv.org/abs/2105.10461