Not OP but I believe the low score is due to not enough training time and incorrect parameters such as frameskip. Space Invaders was mentioned as one of the few games they needed to lower the frameskip (from 4 to 3 or 2?) on because of the flashing lasers. I'm assuming OP left the parameters as is from the implementation by Nathan Sprague, which has frameskip at 4, and trained for a few epochs.
Re:Deep Q-Learning: Space Invaders
by alexlikeits1999 on Monday March 14, 2016 @10:22PM
(#11287031)
Changing the frameskip is not needed any more since the implementation does max of last two frames before processing (same as what DeepMind does)
EDIT: talking about Sprague's implementation btw, not necessarily OPs.
Re:Deep Q-Learning: Space Invaders
by mjaskowski on Tuesday March 15, 2016 @04:36PM
(#11292779)
Actually I did both frame skipping and amxing out two frames as reported in the Natures letter
https://storage.googleapis.com/deepmind-data/assets/papers/D...
Re:Deep Q-Learning: Space Invaders
by mjaskowski on Monday March 14, 2016 @01:11PM
(#11283991)
I noticed that the video does not work in Safari.
Interesting. I am not sure how this deepmind game was played. Note that a typical game with epsilon = 0.1 achieves results around 550 points. In the nature paper they were running the game with epsilon = .05
If you spot any other differences I'd be happy to learn about them.
https://m.youtube.com/watch?v=ZisFfiEdQ_E
For comparison, DeepMind:
https://m.youtube.com/watch?v=ePv0Fs9cGgU
Note that 550 is a very low score in Space Invaders.