birthcert

Yes. Though AI is already in an arms race (mostly US vs. China/Russia).

Likely: Future AI will be decentralized for exactly these reasons. We don't want a single bad actor to control it. Security agencies are now warning that Russia is building a large botnet in the case it needs to go to war, and wants to disable enemy infrastructure. The US has similar needs.

Well designed game theory makes it possible for adversaries to cooperate. So it is no guarantee that Alice is always susceptible to Bob's attacks. Cryptography provides methods that can't be attacked if properly implemented. Defense and offense also can have differing costs: It can be way (computationally) cheaper to create defenses for Alice, than it is to craft adversarial offenses for Bob.

Though the risk is real: Spam preceded spam-filters. There was a short period (in internet years) where spam was more effective than our methods to counter it. So intelligent self-modifying worms/viruses will probably precede intelligent self-learning anti-viruses.

We also see both inverse reinforcement learning (learning about the policy of another agent through observing its behavior), adversarial RL (forcing another trading bot to make unprofitable decisions), and computational arms-races (who has the lowest latency?) between High Frequency Trading firms.

Not necessarily. At a minimum you need access to the sensory environment of the subject: Teens on Twitter are more easily radicalized when their timeline consists largely of terrorist propaganda or war front reporting on civilian casualties. Facebook has done experiments where they changed the sentiment of the timeline for a certain user and saw a significant sentiment change in future posts by that user.

Besides, the average human is not able to set a password, and their brains are open to all sorts of attacks. Cults, terrorist organizations, and multi-level marketing schemes abuse these weaknesses to get their followers to do things that may not be in their own best interest.

You may be interested in the field of adversarial reinforcement learning. In adversarial reinforcement learning, an agent operates in the presence of a destabilizing adversary that applies disturbance forces to its system.

See also the Adversarial Bandit:

> Another variant of the multi-armed bandit problem is called the adversarial bandit, first introduced by Auer and Cesa-Bianchi (1998). In this variant, at each iteration an agent chooses an arm and an adversary simultaneously chooses the payoff structure for each arm. This is one of the strongest generalizations of the bandit problem as it removes all assumptions of the distribution and a solution to the adversarial bandit problem is a generalized solution to the more specific bandit problems.

Good robust RL algorithms are able to learn in the presence of adversarial noise. Correct information is information that allows you to compress reality better. When an agent is able to compress reality better (has access to a better generalizing world model), it will be rewarded. Correct information is information that helps an agent better optimize its policy function.

You actually hit on an interesting angle of research, and you probably will be vindicated in the near future, when adversarial images (those that fool state-of-the-art image classifiers to fail), move to adversarial agents (those that fool other agents into making bad decisions). However, this research was not about multi-agent systems, though the opponents (those that shoot fireballs and try to kill the agent) can already be seen as adversaries to the agent's goal of staying alive longer.

There is a difference between ML research and AI research. AI, traditionally, has more leeway in using intuitive, abstract, or anthropomorphized terms, over ML, which has established learning and optimization theory and a more solid foundation in applied mathematics.

Basically, the deep learning hype in popular science media, owes this status in large part, because it allows nice pictures to be shown. RL research fares well, because they can show video of the agent playing the game. I bet the choice of Doom was also made with this PR in mind, and of course publications like Wired are going to show this work to their readers, over, say, the RELU-paper (impactful in the field, but not much to write an article around).

How did you get into contact with Schmidhuber for co-authoring? What stage was the research at when he joined?

Were you expecting the net to generalize from dream to reality, before you wrote the paper, or did this materialize during experimentation?

Do you expect this approach is also feasible for more difficult games: higher dimensionality, longer delayed rewards?

Both congrats and thanks for writing this very accessible paper. Really found this a creative paper with a lot of inspiration, and the presentation of the results was marvelous.

(BTW: I remember you from the RNN-volleyball game. Back then you had quite some jealous detractors, telling you DeepMind would be too difficult/academic for you. You sure shut those people up!)

I never had an account, and due to this, lost out on many social gatherings (exclusive Facebook) and lost the ability to stay in touch with old friends or get all updates on my family (parents mostly).

The good outweighs the bad for me, but it hurts nonetheless. If you fare better, all the more power to you, but do note: out of sight, out of mind. You may not fully realize what you are giving up by not having a digital social network, from not being able to use Tinder, to more serious stuff, like not knowing a childhood friend has died.

Not being on social media is the equivalent of not having a mobile phone 10 years back. The "essential"-part of Facebook (and other companies, like Google) actually plays a role in European directives. It is why Google has to clean its index on request, and a small niche search engine does not have this responsibility.

Facebook has effectively a monopoly on social networking. Even further: Facebook can be seen as an essential service for maintaining your social life. Not using it puts you at a severe disadvantage.

I don't think monopolies are necessarily a problem that needs fixing, but call it what it is. Facebook makes decisions based on their understanding of dominance.

If you don't like Ford, you can buy a car that you can't drive on most public roads and even fewer gas stations offer you the type of gas you need.