Re:AlphaGo Zero: Learning from scratch
by panic on Wednesday October 18, 2017 @02:01PM
(#15501354)
The core technique of AlphaGo is using tree search as a "policy improvement operator". Tree search doesn't work on most real-world tasks: the "game state" is too complex, there are too many choices, it's hard to predict the full effect of any choice you might make, and there often isn't even a "win" or "lose" state which would let you stop your self-play.
[ Parent
| on HN ]
Re:AlphaGo Zero: Learning from scratch
by habitue on Wednesday October 18, 2017 @02:03PM
(#15501368)
This version explicitly does not use tree search.
MCTS means "Monte-Carlo Tree Search". It's the core of the algorithm. The big difference is that it doesn't use rollouts, or random play: it chooses where to expand the tree based only on the neural network.
Re:AlphaGo Zero: Learning from scratch
by cjbprime on Wednesday October 18, 2017 @02:08PM
(#15501424)
No, 'habitue is correct. This new blog post says that the new software no longer does game readouts and just uses the neural net.
That's not what Monte Carlo Tree search is. The new version is still one neural network + MCTS. There's no way to store enough information to judge the efficiency of every possible move in a neural network, therefore a second algorithm to simulate outcomes is necessary.
Re:AlphaGo Zero: Learning from scratch
by Twirrim on Wednesday October 18, 2017 @02:49PM
(#15501828)
Read the white paper. MCTS is still involved, right the way through.
Re:AlphaGo Zero: Learning from scratch
by ankeshanand on Wednesday October 18, 2017 @02:11PM
(#15501458)
The new version does use MCTS, you should read the paper again. :)
If you read the paper, they do in fact still use monte-Carlo tree search. They just simplify their usage in conjunction with reducing the number of neural networks to 1
Re:AlphaGo Zero: Learning from scratch
by AlexCoventry on Wednesday October 18, 2017 @02:22PM
(#15501576)
It does, during training.
Tree search is also used during play. In the paper, they pit the pure neural net against other versions of the algorithm -- it ends up slightly worse than the version that played Fan Hui, at about 3000 ELO.
Re:AlphaGo Zero: Learning from scratch
by AlexCoventry on Wednesday October 18, 2017 @02:36PM
(#15501724)
Oh, so it's just not using rollouts to estimate the board position? Thanks for the clarification.
It doesn't use rollouts at all:
> AlphaGo Zero does not use “rollouts” - fast, random games used by other Go programs to predict which player will win from the current board position. Instead, it relies on its high quality neural networks to evaluate positions.