We should also be working to secure the channels an AGI might exploit, but most of those are already tied to an economic incentive to invest in security, and are already getting large investments compared to the relatively tiny field of AI safety.
That is precisely the approach that does not make sense to me. By the time friendly developers can launch AGIs, unfriendly developers will not be far behind. And AI safety seems likely to be an issue far before the development of AGI - a non-general malicious AI that can only hack internet services or trick humans into running arbitrary code via conversation is already quite a serious problem.
So to me focusing on "unfriendly developers building narrow AIs" seems more logical than focusing on "friendly developers building AGI".
So I don't think we need to worry about someone maliciously making an evil robot. That's already a problem and we already spent thousands of years figuring out systems of society to protect us from those dangers.
We all know strong AI wouldn't be some sort of robot running around shooting people like a movie, this would be an extinction event worse than a rather large asteroid if it fell into the wrong hands.
The first country or corporation to create strong AI capable of self evolution will either be able to immediately take control of the world as we know it, or worse create something capable of destroying humanity as we know it.
That's not what's going on at all for anyone else that at first glance though that (I'll admit I am guilty of thinking that after glancing at the article as well).
Goal 1: Measure our progress Goal 2: Build a household robot Goal 3: Build an agent with useful natural language understanding Goal 4: Solve a wide variety of games using a single agent
That's all they are trying to accomplish with this (so far anyway). None of which requires strong AI.
TLDR they are formulating some laws of robotics that are a bit more granular for dumb AI which are incapable of self improvement and very task oriented.
In other words the 99% of paperclip optimizers are going to start making virtual paperclips not paving over the universe with actual paperclips. Hacking your reward function is easer than solving hard problems.
PS: Don't do drugs :0
I think making friendly or unfriendly AI is similarily hard (you depend on the same variables just with different THEN clauses).
Making accidentally homicidal well-meaning AI is much easier than both IMHO.
It would still be a "mistake", but not the "oops what does this button do" sort of mistake, but rather the "I could have sworn they fired first" sort of mistake.
That is why I think the main goal of AI safety should be how to defend against intentionally malicious AI. We should be thinking about drone swarms and hacker-AIs rather than paper clips.
The stuff you talk about is farther in the future, and a much tougher problem.
> As long as AI is an open technology there will always be some criminals who just want to see the world burn.
Thankfully we have yet to see terror attacks with nuclear weapons, but the lower barriers of entry for potentially catastrophic AIs undoubtedly is alarming.
I often wonder if analog solutions will not be critical. To continue with your example, a nuke that must be armed with a physical lever would be a rather great hindrance to any AI. Internet communications, etc. will be more tricky, due to the high frequency activity, but the point of having meatspace 'firewalls' on mission critical activities is a nice short term kludge.
Evolution does generate highly optimized systems but generally only when those systems have been around for tens of millions of years. Human-level intelligence has only been around for what, 50k - 100k years? We're probably still more in the 'just works' phase rather than the 'streamlined and optimal' phase.
This distinction is very important when comparing the threat of AI with other significant threats. Before nuclear bombs were built we could not tell you what the difficulty was in creating one. Now that difficulty is well defined, and we can use that knowledge to prevent them from being built by most nations, except the most well funded.
If the barrier for entry for AGI (then ASI) is lower than we expect, then the threat of AI is significantly different than if AGI/ASI can only be created by nation states.
The reason I am framing things this way is we need to be very careful here because we are starting to turn towards speculation.
Science fiction turns out to be true when physical reality agrees that it can be true. This again, is why we have a global communications network and personal wireless devices connected to it. This is also the reason we do not go faster than light.
The reason we don't have flying cars is they are completely possible. They are also terribly dangerous and expensive and a complete waste of energy.
The reason we don't have AGI is not that it is impossible, again if nature can create it, we can recreate it. Since we don't have a good understanding of the networked nature of emergent intelligence we cannot create a power optimized network that would allow us to create a energy efficient version. AGI itself is a complete waste of energy at this point. We already have many types of AI that are energy efficient and used in products now.
This is a ridiculous argument. Furthermore, even if it were true, it tells us nothing about the timeline. It could take 10,000 years for all we know.
I don't think you need AGI to cause a catastrophe. A narrow AI specializing in cyberattacks could be catastrophic, and is probably possible with current techniques.
Access to that kind of data would help an AI determine the people that are more susceptible to manipulation. Add in records on health care, as you mentioned, and information on debts and you have data that can help an AI gather as many human minions as it needs.
Maybe. Can you make a human that can't be suborned by a superhuman evil AI?
That's how cold war remained cold.
IMO, a better approach to AI safety research is to focus on securing the first channels that a malicious AI would be likely to exploit. Like spam, and security. Can you make communications spam-resistant? Can you make an unhackable internet service?
Those seem hard, but more plausible than the "Watch out for paperclip optimizers" approach to AI safety. It just feels like inventing a way to build a nuclear weapon that can't actually explode, and then hoping the problem of nuclear war is solved.