Agent57: Outperforming the human Atari benchmark

Re:Agent57: Outperforming the human Atari benchmark by WnZ39p0Dgydaz1 on Tuesday March 31, 2020 @09:15AM (#22737167)

I agree with you overall, but while it was relatively simple, DQN was never "elegant". The elegant and most important part of DQN was to use a CNN for learning a good image representation - but CNNs are completely unrelated to RL.

The sole contribution of DQN was to introduce two ugly band-aids, experience replay and a target network, to make the CNN training stable in an RL setting.

So, DQN = CNN (elegant and simple) + ugly band aid

[ Parent | on HN ]

Re:Agent57: Outperforming the human Atari benchmark by pabloem on Tuesday March 31, 2020 @11:09AM (#22738234)

Just a side note. Probably not very fair to attribue the introduction of experience replay to DQN authors. This technique was popularized in 1992 by Lin[1] and used later in many works to improve performance. The difference, perhaps, is that in DQN, experience replay is not only a way to improve performance, but it's a must to obtain stability.

[1] http://www.incompleteideas.net/lin-92.pdf

[ Parent | on HN ]