Re:More on Dota 2
by mannykannot on Thursday August 17, 2017 @06:40AM
(#15035433)
> The key becomes finding the right scoring algorithms to tweak the neural net correctly towards the desired outcome.
Does this not become something similar to supervised learning if you are scoring internal states of the game? (i.e. scoring on more than just the outcome and things that violate the rules?)
I don't know what they're scoring on; they've not elaborated on that. However, when I first read into OpenAI it looked like many of their game links were to simply view the screen. To that end, it wouldn't surprise me if they're identifying things you'd otherwise visually see in the game and simply working off them. To use internal states not immediately visible to the player would be sort of disingenuous IMO.
Why wouldn't algorithm reach a local maxima when playing with itself, or even degrade over time by opening up to unknown attacks?
One typically keeps pools of trained networks to combat this.
My understanding, from talking to a ML friend this morning, is that the latest progress is taking reinforcement learning and applying deep learning approaches (nets, etc.) to it. The key becomes finding the right scoring algorithms to tweak the neural net correctly towards the desired outcome.
The self-play really is the reinforcement side of things at work. How you take that 'score' and use it to correctly modify the input weightings - be them in a neural net, traditional algorithm, etc. - is the key.