News for hackers. Stuff that matters.
Re:Ask HN: What is the apocolyptic scenario for AI “breaking... by musicale on Thursday April 27, 2023 @05:06PM (#35734603)
Harris and Raskin [1, via 2] note the following risks:

    Reality collapse
    Fake everything
    Trust collapse
    Automated loopholes in law / fake religions / cyberweapons / lobbying
    Automated fake religions
    Exponential blackmail / scams
    A-Z testing of everything
    Synthetic relationships
    AlphaPersuade
They also note potential privacy risks (ai-augmented ambient wifi to see through walls, ai-enabled fmri-based brain reading) as well as the potential risk of assisting criminals and terrorists (e.g. AI approaches research-level chemistry, tells users how to make a nerve agent from materials they buy at home depot.)

A more subtle risk they point out is theory of mind, which is that with an adequate theory of mind the AI system may act in highly manipulative ways. We may be seeing this already with synthetic relationships.

They also point out that AI-fueled malware creation is already a thing.

Another worrisome scenario is that models seem to be able to be optimized and shrunk. The emergent behavior from millions of them is unclear.

And they note that AI (unlike nukes) can improve itself. This may lead to surprising step functions in emergent capabilities at an increasing rate, with unpredictable and likely unintended consequences.

[1] https://www.youtube.com/watch?v=xoVJKj8lcNQ (video) https://www.humanetech.com/podcast/the-ai-dilemma (podcast and transcript)

[2] https://mleverything.substack.com/p/thought-on-ai-dilemma-pa...

There are no comments on this comment yet.