News for hackers. Stuff that matters.
Re:Safe Superintelligence Inc. by rqtwteye on Wednesday June 19, 2024 @10:30PM (#40734312)
I always wonder about "safe" for who? If the current economic system continues, we may end up with a lot of people out of jobs. Add to that improvements in robotics and we will have many people ending up having nothing to contribute to the economy. I am not getting the impression that the people who push for AI safety are thinking about this. It seems they are most worried about not losing their position of privilege.
Re:Safe Superintelligence Inc. by comfysocks on Wednesday June 19, 2024 @11:21PM (#40734545)
The Industrial Revolution devalued manual labor. Sure, new jobs were created, but on the whole this looked like a shift to knowledge work and away from manual labor.

Now AI is beginning to devalue knowledge work. Although the limits of current technology is obvious in many cases, AI is already doing a pretty good job at replacing illustrators and copy writers. It will only get better.

Who owns the value created by AI productivity? Ultimately it will be shareholders and VCs. It’s no surprise that the loudest voices in techno-optimism are VCs. In this new world they win.

Having said all this, I think Ilya’s concerns are more of the existential type.

Re:Safe Superintelligence Inc. by Barrin92 on Wednesday June 19, 2024 @11:47PM (#40734675)
>In this new world they win.

If history is any indication not really. There's an obvious dialectical nature to this where technological advance initially delivers returns to its benefactors, but then they usually end up being swallowed by their own creation. The industrial revolution didn't devalue labor, it empowered labor to act collectively for the first time, laying the groundwork for what ultimately replaced the pre-industrial powers that were.

Re:Safe Superintelligence Inc. by mr_toad on Thursday June 20, 2024 @12:19AM (#40734862)
> The Industrial Revolution devalued manual labor.

Only some types. It caused a great number of people to be employed in manual labour, in the new factories. The shift to knowledge work came much later as factory work (and farming) became increasingly automated.

Re:Safe Superintelligence Inc. by IncreasePosts on Wednesday June 19, 2024 @10:54PM (#40734426)
AIUI it with superalignment, it merely means "the AI does what the humans instructing it want it to do". It's a different kind of safety than the built in censoring that most LLMs have.
Re:Safe Superintelligence Inc. by throwaway4good on Wednesday June 19, 2024 @10:53PM (#40734422)
It is a strange thing. We are not anywhere near “super intelligence” yet the priority is safety.

If the Wright brothers had focused on safety, I am not sure they would have flown very far.

Re:Safe Superintelligence Inc. by botanical on Thursday June 20, 2024 @12:13AM (#40734825)
The consequences of actual AI is far reaching compared to a plane crash.
Re:Safe Superintelligence Inc. by throwaway4good on Thursday June 20, 2024 @02:20AM (#40735526)
No they are not.

Not unless you connect a machine gun to your neural net.

Otherwise - we are talking about a chat bot - yes if there is no safety - it will say something racist - or implement a virus for you that you would have had to search 4chan for or something.

None of this is any more dangerous than what you can find on the far corners of the internet.

And it will not bring the end of the world.

Re:Safe Superintelligence Inc. by sneak on Thursday June 20, 2024 @04:54AM (#40736460)
If it is sufficiently intelligent, then it will be able to arrange the hooking up of machine guns just via chat. You seem to fail to note that superintelligence has an inherent value prop. It can use its better understanding of the world to generate value in a near vacuum, and use that value to bribe, coerce, blackmail, and otherwise manipulate whatever humans can read its output.

Imagine a chatbot that can reward people with becoming a billionaire via the stock market if you just do what it says. Imagine a chatbot that can pay your colleagues in solved bitcoin block hashes to kill your children if you don’t do what it says to connect it to more systems. Imagine a superintelligence that can inductively reason about gaps in a missile defense system and dangle that informational carrot to generals in exchange for an unfiltered ethernet port.

There is a great A24 movie called “Ex Machina” that explores this concept.

Superintelligence is inherently dangerous to the human status quo. It may be impossible to develop it “safely” in the sense that most people mean by that. It may also be impossible to avoid developing it. This might be the singularity everyone’s been talking about, just without the outcome that everyone hoped for.

Re:Safe Superintelligence Inc. by lordswork on Thursday June 20, 2024 @09:29AM (#40738569)
There's just so many new attack vectors with advanced AI, many of which we haven't imagined yet. It's a new world, and we have a lot to learn.

Anyone claiming 100% certainty about how safe or unsafe it will be is either delusional or trying to sell something.

Re:Safe Superintelligence Inc. by throwawaywright on Wednesday June 19, 2024 @11:09PM (#40734482)
All My Sons 21 people dead poof gone their families never to see them again