News for hackers. Stuff that matters.
Re:Deep Mind Playing Montezuma's Revenge with Intrinsic Moti... by gwern on Wednesday June 08, 2016 @12:20PM (#11863336)
The ALE agents are almost always using grey-scale pixels in the first place (to save on processing and NN space requirements), so flipping the color makes no difference.
Re:Deep Mind Playing Montezuma's Revenge with Intrinsic Moti... by seanwilson on Wednesday June 08, 2016 @12:28PM (#11863407)
Is it learning the specific level layout or can it adapt when the screens are different and order swapped?
Re:Deep Mind Playing Montezuma's Revenge with Intrinsic Moti... by gwern on Wednesday June 08, 2016 @03:39PM (#11864990)
DQN is stateless, so at every point it's reacting to just the current frame (technically, the average of the last 4 frames IIRC) being fed into a CNN and outputing a motion. It doesn't care about ordering all that much because it has no memory. In this case, it's probably learning stuff like 'if there's a key, go towards it' and 'if you have a key, go towards a barrier'. If each screen keeps working the same way, then I guess it would achieve about the same performance as it does now. New screens with new layouts/patterns of enemies will test how well its learned heuristics generalize, though.

Personally, I think that someone should be trying a DQN with an RNN rather than CNN in it to see if that helps on the harder levels. Or better yet, combine it with some of the memory mechanisms and see if it can start doing some real long-term planning.

Re:Deep Mind Playing Montezuma's Revenge with Intrinsic Moti... by SandB0x on Wednesday June 08, 2016 @04:55PM (#11865588)
I believe the input state is the past 4 frames stacked or concatenated.