News for hackers. Stuff that matters.
Re:Human-level control through deep reinforcement learning by bmh100 on Wednesday February 25, 2015 @04:55PM (#9109964)
> ...the authors used the same algorithm, network architecture, and hyperparameters on each game...

This is huge. It shows that the algorithm was able to generalize across multiple problem sets within the same domain of "playing Atari 2600 games", and not simply a "lucky" choice of algorithm, network architecture, or hyperparameters that a random search for each game might choose. This is also not a violation of the No Free Lunch (NFL) Theorem [1], because the domain is limited to playing Atari 2600 games, which share many characteristics.

[1]: https://en.wikipedia.org/wiki/No_free_lunch_in_search_and_op...

Re:Human-level control through deep reinforcement learning by chriswarbo on Wednesday February 25, 2015 @07:47PM (#9110741)
The No Free Lunch Theorems don't apply in reality anyway, as explained at https://en.wikipedia.org/wiki/No_free_lunch_in_search_and_op...

Essentially: NFL says the performance (1 / number of trials are required to find a solution), when averaged over all fitness functions, will be the same for all optimisation algorithms. However, most fitness functions require more computational power than the Universe to evaluate; making even a single trial impossible to perform.

Since those problems are already impossible, it doesn't matter how many trials our optimisation algorithm would require when tackling them. We can use this unobservable bad performance to offset some observable good performance, to remain within the average dictated by NFL whilst still becoming objectively better.

In fact, one of this paper's authors (Shane Legg) has used this fact to define a general intelligence test which favours performance on simple functions http://arxiv.org/abs/0712.3329

Re:Human-level control through deep reinforcement learning by bsdetector on Wednesday February 25, 2015 @07:35PM (#9110702)
> the algorithm was able to generalize across multiple problem sets

Did it really? I think they reset it and retrained it for each game.

I'd like to know how much more is needed to make one instance of the AI that can successfully play any of the games. To play all 49 games that it could learn, does it need to be an extra level deep? Or 49 times larger? Or 2^49 times more?

Re:Human-level control through deep reinforcement learning by DavidSJ on Wednesday February 25, 2015 @09:15PM (#9110944)
What they mean by "the algorithm" is the learning algorithm, not the actual learned behavior. Historically in machine learning a great deal of hand tuning of learning algorithms is required for them to even be effective on a single problem, so it's impressive that a single algorithm was able to learn to play many different games effectively.

Having a single instance of an algorithm learn to perform well on many different tasks is really a separate problem than what they're dealing with here.

Re:Human-level control through deep reinforcement learning by ZeroFries on Wednesday February 25, 2015 @08:37PM (#9110866)
I think you could establish an upperbound at 50 times larger, because each set of "neurons" could be completely independent for each game, with 49 neurons to determine which set to use.

If they have similar features (which they undoubtedly would), the size could drop a lot.

Of course, I have no training in neural nets, and so my conclusions are reached from general understanding/reasoning.

Re:Human-level control through deep reinforcement learning by meric on Thursday February 26, 2015 @04:54AM (#9111815)
Can one human successfully play all of the games without prior practice in each game? As far as I know a human has to practice almost every game to be able to play all of them without losses. I think for an AI, achieving this standard is a good result - first practicing in each game and then play through all of them without losses.
Re:Human-level control through deep reinforcement learning by bsdetector on Thursday February 26, 2015 @09:45AM (#9112725)
A person that learns 1 game will learn the next game much faster, because they have learned a concept such as a bullet or switch or reflection or wrapping. We take this for granted, but there was a time when Breakout was actually marginally fun because it was new.

A person that's played all the other games in the list can win Montezuma's Revenge on the first try; this AI can't play Montezuma's Revenge at all.