mjaskowski
https://storage.googleapis.com/deepmind-data/assets/papers/D...
The main trick to notice is that you can't provide consecutive frames as mini-batches as these would be highly correlated and would derail stochastic gradient descent.
So we keep many frames (and all other necessary information) in memory and draw these experiences uniformly to form a minibatch that becomes input to the neural network
and a more recent and more detailed: https://storage.googleapis.com/deepmind-data/assets/papers/D...
You usually think of Q as a function (S, A) -> (Expected accumulated future reward)
which is equivalent to S -> A -> (Expected accumulated future reward)
the Neural Network is S -> (A -> (Expected accumulated future reward)) or if you whish the output layer of neural network consists of |A| neurons. Each indicates the (Expected accumulated future reward) given current experience.
Interesting. I am not sure how this deepmind game was played. Note that a typical game with epsilon = 0.1 achieves results around 550 points. In the nature paper they were running the game with epsilon = .05
If you spot any other differences I'd be happy to learn about them.