Re:Deep Reinforcement Learning to Play Space Invaders [pdf]
by gwern on Monday April 10, 2017 @12:24PM
(#14079874)
So... the original DQN paper already did Space Invaders as part of the ALE environment, getting better than their human player's score. The DeepMind source code as well as other reimplementations have been out for years, and likewise for Double Q-learning, so it didn't really need someone to reimplement it either. Looking at this paper, I'm not sure what is new or interesting, other than the experiment with using the Atari RAM as features (but even that doesn't seem as interesting as the paper which used MCTS on the RAM: "Deep learning for real-time Atari game play using offline Monte-Carlo tree search planning", Guo et al 2014 http://papers.nips.cc/paper/5421-deep-learning-for-real-time... ). Or is this just the report from an (impressive) student project and not intended to be groundbreaking research?
Re:Deep Reinforcement Learning to Play Space Invaders [pdf]
by alexlikeits1999 on Monday April 10, 2017 @10:58PM
(#14084503)
Just a nitpick, but the original didn't do better than human scores. It does better than their human panel, who seem to suck at space invaders and a whole bunch of other games. In general DQN does worse than expected at space invaders for some reason (compared to, say, galaxian, a very similar game, at which it does very well)
They didn't even bother to do something original except for using the RAM state instead of pixel values like the original paper. It's something that's easily doable with the openAI gym. This seems like a course project at stanford.
Re:Deep Reinforcement Learning to Play Space Invaders [pdf]
by tMMkeJRMD0 on Tuesday April 11, 2017 @12:31AM
(#14084857)
Hi, one of the authors here. Yes this is a school project. This pdf was a draft, not the final version for the class submission (the paragraph in question never made it to the final version). Don't know who shared the link here, but it is updated now.
Re:Deep Reinforcement Learning to Play Space Invaders [pdf]
by ramblenode on Monday April 10, 2017 @11:28PM
(#14084603)
For convenience.
Original:
However reinforcement learning presents several challenges from a deep learning perspective. Firstly, most successful deep learning applications to date have required large amounts of hand-labelled training data. RL algorithms, on the other hand, must be able to learn from a scalar reward signal that is frequently sparse, noisy and delayed.
New:
However reinforcement learning presents challenges from a deep learning perspective - most supervised deep learning applications require large amounts of labeled training data whereas reinforcement learning algorithms must be able to learn from scalar rewards that are often delayed.
Immediately after the plagiarized section is a citation to the paper which was plagiarized (Mnih et al.).
Re:Deep Reinforcement Learning to Play Space Invaders [pdf]
by MichailP on Monday April 10, 2017 @04:56PM
(#14082473)
Off topic but related to deep learning. Did anybody encounter use of machine learning in mesh generation, as in 2D, 3D triangular, quad, hex mesh generation?
Re:Deep Reinforcement Learning to Play Space Invaders [pdf]
by HammadB on Monday April 10, 2017 @05:25PM
(#14082709)
I am also interested in this. I've seen work done in generative models for voxels but nothing in meshes unfortunately. I also haven't looked quite that hard.
Third paragraph of Intro in original DeepMind paper https://arxiv.org/pdf/1312.5602.pdf
&
Second paragraph of Intro(5th line) of this project. http://nihit.github.io/resources/spaceinvaders.pdf
They didn't even bother to do something original except for using the RAM state instead of pixel values like the original paper. It's something that's easily doable with the openAI gym. This seems like a course project at stanford.
Original:
However reinforcement learning presents several challenges from a deep learning perspective. Firstly, most successful deep learning applications to date have required large amounts of hand-labelled training data. RL algorithms, on the other hand, must be able to learn from a scalar reward signal that is frequently sparse, noisy and delayed.
New:
However reinforcement learning presents challenges from a deep learning perspective - most supervised deep learning applications require large amounts of labeled training data whereas reinforcement learning algorithms must be able to learn from scalar rewards that are often delayed.
Immediately after the plagiarized section is a citation to the paper which was plagiarized (Mnih et al.).