Preparing for Malicious Uses of AI

Re:Preparing for Malicious Uses of AI by ynniv on Thursday February 22, 2018 @10:50AM (#16438345)

The "best" part is how good an AI would be at privilege escalation. Go ahead, build out that security camera network, automate your cars, store your money using cryptography. I'm a geek and I love the way technology increases our collective prosperity, but most of our security relies on the attackers being approximately as smart as any other human. To a sufficiently capable intelligence, encrypted machine code is just as understandable as the most elegant source code. Once you fully understand something, anticipated edge cases are easy to find and exploit... every day we expand the capabilities of a future strong AI.

[ Parent | on HN ]

Re:Preparing for Malicious Uses of AI by gone35 on Thursday February 22, 2018 @01:24PM (#16439919)

To a sufficiently capable intelligence, encrypted machine code is just as understandable as the most elegant source code.

Depends on what you mean by "sufficiently" and "encrypted"!

I'm not an expert, but my guess is these are still truly difficult, deep unsolved questions... We don't even know yet if one-way functions exist; and existing modes of analysis increasingly seem insufficient to fully account for the apparent recent successes of this last wave of 'AI' on ostensibly non-convex optimization problems! (Not to mention the debates about the possibility of fundamental physical/computational complexity limits to intelligence explosion, and so forth...)

So who knows at this point; but I think it is still an open question whether a future AI could reasonably 'break' even today's commodity encryption.

[ Parent | on HN ]

Re:Preparing for Malicious Uses of AI by ynniv on Thursday February 22, 2018 @03:51PM (#16441270)

The problem is that we don’t know whether our encryption is vulnerable to advanced attacks. Cryptographic hash functions actually have a useful lifespan based on the math and the cost of computing. The more time we spend analyzing the math, the more likely we are to find unexpected weaknesses.

http://valerieaurora.org/hash.html

[ Parent | on HN ]

Re:Preparing for Malicious Uses of AI by breuleux on Thursday February 22, 2018 @06:45PM (#16442709)

It depends on the rate of improvement of AI and what the intrinsic strength of defence is against attack. Arguably, security will start incorporating AI techniques, and by the time AI attackers surpass human capability, AI defenders would likely be too strong for humans, and quite possibly strong enough to hold against a higher intelligence.

[ Parent | on HN ]

Re:Preparing for Malicious Uses of AI by adamsea on Thursday February 22, 2018 @04:31PM (#16441580)

"Kuang Grade Mark Eleven did the Times ICE like a hot titanium wire through a cube of chilled petroleum jelly," smirked the Finn.

[ Parent | on HN ]

Re:Preparing for Malicious Uses of AI by nyolfen on Thursday February 22, 2018 @11:50AM (#16438951)

this comment is incoherent

[ Parent | on HN ]

Re:Preparing for Malicious Uses of AI by tree_of_item on Thursday February 22, 2018 @12:05PM (#16439096)

Why? It's pretty simple, if we computerize everything a future AI is going to have more leverage over the world. If you believe Cory Doctorow when he says that everything is a computer, then everything is vulnerable to future AI.

[ Parent | on HN ]

Re:Preparing for Malicious Uses of AI by book_mentioned on Thursday February 22, 2018 @11:55AM (#16438999)

My takeaway/TL;DR: OpSec is harder vs. AI (ever-vigilant pattern matching) than humans.

But it did drift into discussing a more offensive AI than passive monitoring done today.

[ Parent | on HN ]

Re:Preparing for Malicious Uses of AI by ynniv on Thursday February 22, 2018 @12:44PM (#16439489)

How many sentences is TL these days? But yes, this is an offensive AI, and no, we aren’t passive-only today - look at the IoT botnets.

[ Parent | on HN ]