News for hackers. Stuff that matters.
Re:Human-level control through deep reinforcement learning by Someone on Wednesday February 25, 2015 @07:39PM (#9110719)
For comparison: http://www.cs.cmu.edu/~tom7/mario/. That is way more of a hack, but I am not sure this is that big a step forward. Space invaders and breakout aren't the hardest games and I haven't heard a hard argument that it is just a matter of scale to create a machine that, say, plays chess.
Re:Human-level control through deep reinforcement learning by TheEzEzz on Wednesday February 25, 2015 @08:28PM (#9110849)
The biggest differences are this:

1. The Mario algo has direct access to the game state, and will only work for games where it has that game state access. The DeepMind algo plays directly from the screen pixels. That means DeepMind has to first learn to interpret time varying (!) visual information correctly, then deduce rules and good play strategy on top of that leaky abstraction. That's hard. It also means the algorithm can be applied to any game with a screen output, not just to an Atari.

2. The Mario algo is doing a direct search through move space. It can back up and explore a different branch of the tree and play differently to see a different outcome. When the DeepMind algo plays Atari, it can't undo a move that it just did. It has to make good choices, using intelligence, just like a human player would.

The impressive thing here is not that it plays Atari games. You're right, we have had AIs that can do this for a long time, even better than this. The impressive thing is that it's a single AI algorithm that works for many games, and that is learning directly from the screen. We have not had anything like this before.

Re:Human-level control through deep reinforcement learning by chriswarbo on Wednesday February 25, 2015 @08:13PM (#9110812)
That's pretty cool (especially the ending of Tetris!), but the crucial difference is that it's trying to learn by observing what happens during a known-good run, then it sounds like some game-specific learning is performed. In contrast, the Arcade Learning Environment (the benchmark used in TFA) is completely autonomous; there is no human guidance or game-specific learning: anything it learns must work well across all of the games (ie. it's either generally useful, or it gets surrounded by a bunch of conditionals).

In particular, the author of your link says he's surprised that the algorithm can sometimes get further than twice the length of his pre-recorded training sequence.