News for hackers. Stuff that matters.
Re:AlphaGo Zero: Learning from scratch by aeleos on Wednesday October 18, 2017 @02:38PM (#15501737)
The fact that they only used self play with no outside input here is really interesting. I wonder if this system produced more new styles of play. While I am not that familiar with Go, I know in some of the other articles they talk about things like Chinese starts that are specific to certain cultures. I wonder if the fact that it had no outside reinforcement made it produce movements that we have already seen that are somehow inherent to the game, or if it produced many more new moves that were a result of it learning without any possibility of cultural interference. According to the article it did invent some unconventional and creative moves, but I also wonder how much it rediscovered.

I also wonder how much it’s style of play changes if it were re trained, due to the random start that it is given. Maybe that would produce something like seeds for procedurally generated worlds in games. Like if they could find a seed for a Chinese or japanese players, or ones that more aggressive styles. This is some pretty cool work and may open up even more doors for pure reinforcement learning

Re:AlphaGo Zero: Learning from scratch by kndyry on Wednesday October 18, 2017 @04:27PM (#15502674)
I don't think it's an overstatement to say that, since playing Lee Sedol in 2016, AlphaGo has completely revolutionized professional and amateur go. It's certainly not unprecedented — the last major revolution happened in the early 20th century (often called the 'Shin Fuseki' era [0]) — but AlphaGo has demonstrably surpassed any previous high-water mark.

> I wonder if this system produced more new styles of play.

Absolutely. One such innovation has been the use of early 3-3 invasions [1]. There are many more, and indeed AlphaGo's games are still being analyzed by professional players. Michael Redmond, a 9-dan professional, has been working with the American Go Association on one such series [2].

> I wonder if the fact that it had no outside reinforcement made it produce movements that we have already seen that are somehow inherent to the game...

Interestingly, yes. Strong players have commented that AlphaGo seems to agree with things that players like Go Seigen [3] have suggested in the past, but that were never fully developed or understood [4].

Very, very interesting work indeed.

[0] https://senseis.xmp.net/?ShinFuseki

[1] https://www.eurogofed.org/index.html?id=127

[2] http://www.usgo.org/news/category/go-news/computer-goai/mast...

[3] https://senseis.xmp.net/?GoSeigen

[4] https://lifein19x19.com/forum/viewtopic.php?f=13&t=14129

Re:AlphaGo Zero: Learning from scratch by alex_stoddard on Wednesday October 18, 2017 @04:46PM (#15502863)
I have only skimmed the paper but one thing I don't see any discussion of is whether komi (the handicap given to white for going second) is correct.

They do say the rules used for all games, including self-play, set komi consistently to 7.5 .

If the strongest AI was consistently winning predominantly with one color it would be an indication that komi isn't fair for the best play.

Of the 20 games released for the strongest play it appears white won 14 times and black 6. I don't think that is enough to be conclusive but maybe komi is too high.

I wonder if different "correct" play at the strongest levels would be learned with a 6.5 komi.

Re:AlphaGo Zero: Learning from scratch by gizmo686 on Wednesday October 18, 2017 @04:57PM (#15503000)
You can only change komi by full point increments. There is a .5 to break ties, but a komi of 7.5 is identical to one of 7.4.

From a theoretical standpoint, any non-integer komi should lead to one player winning 100% of the time. So even if the actual win ratio is 14:6 at komi=7.5 that might still be the best value.

Re:AlphaGo Zero: Learning from scratch by btilly on Wednesday October 18, 2017 @06:32PM (#15503845)
If you had an estimate of the real difference, you could switch to breaking ties randomly. Black wins 60% of the ties, white wins 40%. There will be a ratio at which each side should win 50% of the time.

I agree that with perfect play, it will be a 50% of a tie to each side. But it is still interesting to ask for a better estimate of practical play.

Re:AlphaGo Zero: Learning from scratch by xelxebar on Wednesday October 18, 2017 @08:51PM (#15504589)
Michael Redmond mentions this in the AlphaGo vs AlphaGo review series he's doing with the AGA. AlphaGo selfplay games are with 7.5 komi under Chinese rules, and apparently, Deepmind has stated that black vs white wins is almost exactly 50/50. IIRC Redmond mentioned that white (?) only had some sub 1% advantage in the entire self-play corpus.
Re:AlphaGo Zero: Learning from scratch by Jach on Wednesday October 18, 2017 @05:04PM (#15503076)
I don't remember where I read it but in some earlier versions of AlphaGo they tried a komi of 6.5 and black ended up winning more often. That indicates the correct komi value is 7, but since Go doesn't have ties, you have to pick which side you want to favor to break the tie. (White seems reasonable.)
Re:AlphaGo Zero: Learning from scratch by conistonwater on Wednesday October 18, 2017 @06:40PM (#15503901)
That doesn't sound right because in Chinese rules, which is what AG uses, komi only changes in steps of two. Are you sure about 6.5? Could it be 5.5?
Re:AlphaGo Zero: Learning from scratch by Jach on Wednesday October 18, 2017 @07:50PM (#15504313)
Well in Japanese rules the komi is 6.5 so that's the alternative that tends to come up. Some quick searching I found a transcript from one of the games where DeepMind said 7.5 slightly favors white but they didn't say anything about 6.5 or 5.5, while a random comment from r/baduk claims that pro game analysis shows 6.5 slightly favors black and 7.5 slightly favors white.
Re:AlphaGo Zero: Learning from scratch by wetpaws on Friday October 20, 2017 @05:49PM (#15519337)
I guess they should then randomize komi so any player would have 50/50 chance of getting a small edge.
Re:AlphaGo Zero: Learning from scratch by fhe on Thursday October 19, 2017 @12:41AM (#15505525)
the correct komi number has puzzled Go players for centuries, now we might finally have a chance to figure out the right answer (although not without some reservations). over the last 5 decades, komi has consistently been raised to keep the game more leveled between white and black (black makes the first move, so has the advantage). historically, there was no komi, and people kept an even game by always playing even number of games with each player switching sides after each game.

for whatever reason, it's no longer feasible in modern pro game (not to mention that this could result in no winner if each player wins half the game), so komi was introduced. at first at 5.5, and steadily climbed higher to 7.5 at present. In pro game, even a change of 1 is considered a big deal, so from 5.5 to 7.5 is hardly trivial.

Now with alphago playing "perfect" games against itself, we might finally be able to put to rest the debate of the correct komi (the Japanese Go associations for decades have kept meticulous records of every professional game, in order to find the correct komi).

There is a big "but" though. The correct komi at Alphgo Zero's level might not be the correct komi for human level players (AlphaGo is estimated to be 2-3 handicaps above human play; this is a bigger gap between the average pro player and the best amateurs).

Indeed, the change from 5.5 komi to 7.5 komi also had a lot to do with the change in play style rather than simply zooming in on the "correct" komi number. In the 70s and 80s, predominant play style was more conservative, and 5.5 might well be the correct komi for the time (defined as resulting in 50:50 chance of winning for either side). As play style shifted to become more aggressive and confrontational (actually fueld somewhat by the introduction of komi), it was discovered that komi needs to be raised to keep chances of winning at 50:50.

To make an analogy, suppose one is playing a casino game of chance that gives the house a slight advantage (similar to the first mover advantage for black in go). If one only makes small bets, the house will end up winning only a small amount. in other words, the player needs to be compensated by a small amount to make the game "fair".

If however one makes big bets (i.e. more aggressive game play), then the compensation needs to be bigger too, to make the game "fair", even if the underlying probabilities have not changed.

following this logic, while 7.5 komi is fair for Alphago vs. alphago games, it might not be the right number for human games. I suspect it might be samller for humans.... if only we could calibrate Alphago to the average human level and generate millions of self-play games...

Re:AlphaGo Zero: Learning from scratch by maxerickson on Wednesday October 18, 2017 @07:03PM (#15504038)
They could just test it right?

Or am I misunderstanding the hardware requirements?

Re:AlphaGo Zero: Learning from scratch by Spare_account on Friday October 20, 2017 @07:47AM (#15514985)
With respect to your very interesting comment (I genuinely appreciate your input), you appear to have mis-understood the comment you were replying to.

You've commented on the differences in the style of play that AlphaGo introduced, but the post you were replying to (by aeleos) was going a step further and hypothesising about the potential for a newer, completely 'non-human' style that AlphaGo Zero may have created.

Your comments definitely contribute to the discussion but it was bugging me that there appeared to be a tangent forming about AlphaGo that was overlooking AlphaGo Zero which would be the more interesting area to explore.

Re:AlphaGo Zero: Learning from scratch by posterboy on Friday October 20, 2017 @07:36PM (#15519828)
"more new styles of play" seems to indicate that non-human play.
Re:AlphaGo Zero: Learning from scratch by Spare_account on Saturday October 21, 2017 @11:30AM (#15522156)
Yes, aeleos was interested in that, and so am I, and it seems to be what this entire thread _should be about_. kndyry steered back towards AlphaGo. I'm not sure this merits any further disection.
Re:AlphaGo Zero: Learning from scratch by shmageggy on Wednesday October 18, 2017 @03:02PM (#15501951)
The key part from the paper:

> To assess the merits of self-play reinforcement learning, compared to learning from human data, we trained a second neural network (using the same architecture) to predict expert moves in the KGS Server data­ set; this achieved state-of-the-art prediction accuracy compared to pre­ vious work 12,30–33 (see Extended Data Tables 1 and 2 for current and previous results, respectively). Supervised learning achieved a better initial performance, and was better at predicting human professional moves (Fig. 3). Notably, although supervised learning achieved higher move prediction accuracy, the self-learned player performed much better overall, defeating the human-trained player within the first 24 h of training. This suggests that AlphaGo Zero may be learning a strategy that is qualitatively different to human play.

Re:AlphaGo Zero: Learning from scratch by aeleos on Wednesday October 18, 2017 @03:41PM (#15502258)
That is really interesting. Given a neural network that solely exist to play Go, one that is influenced by the human mind is limited compared to the exact same set of neurons that doesn't have that influence.

EDIT: changed a set of neurons to neural network per andbbergers comments

Re:AlphaGo Zero: Learning from scratch by andbberger on Wednesday October 18, 2017 @04:06PM (#15502465)
Please don't refer to it as 'a set of neurons' - it only serves to fuel the (IMO) absolutely ridiculous AI winter fearmongering, and is also just a bad description. Neural nets are linear algebra blackboxes, the connections to biology are tenuous at best.

Sorry to be that guy, but the AI hype is getting out hand. COSYNE this year was packed with papers comparing deep learning to the brain... it drives me nutty. Convnets can be reasonably put into analogy with the visual system.... because they were inspired from it. But that's about it.

To address your actual comment: I would argue that this is not really interesting or surprising (at least to the ML practitioner), it is very well known that neural nets are incredibly sensitive to initialization. Think of it like this: as training progresses, parameters of neural nets move along manifolds in parameter space, but they can get nudged off of the "right" manifold and will never be able to recover.

Sorry for the rant, the AI hype is just getting really out of hand recently.

Machine learning is specifically not magic. Blackboxes are not useful. Convnets work so well because they build the symmetries of natural scenes directly into the model - natural scenes are translation invariant (as well as a couple of other symmetries), anything that models them sure as hell better have those symmetries too, or you're just crippling your model with extra superfluous parameters.

Re:AlphaGo Zero: Learning from scratch by aeleos on Wednesday October 18, 2017 @05:11PM (#15503151)
I changed my comment to neural network since a set of neurons is somewhat wrong, but I don't really agree that there isn't much of a connection between this and biology. While there might not be much of a connection between how they currently work and how our brains work, the whole point of machine learning and neural networks is to improve computers performance on the things we are good at. And while originally it was loosely modeled on it, and might be different know, it doesn't make it so people can't compare it to the brain. It would be wrong to say it is exactly like the brain, but I don't think there is anything wrong with comparing and contrasting the two. If our goal is to improve performance and we are the benchmark, then why shouldn't we compare them.

What I found interesting was mainly that it was us who nudged the parameter space you talked about into the "wrong" one manifold, especially given how old and complicated Go is. The sheer amount of human brain power that has been put into getting good at a game wasn't able to find certain aspects of it, and in 60 hours of training a neural network was able to.

Re:AlphaGo Zero: Learning from scratch by andbberger on Wednesday October 18, 2017 @08:35PM (#15504536)
I'm not saying there is nothing of value to be obtained by investigating connections between ML and the brain. That's how I got into ML in the first place, doing theoretical neuro research.

We absolutely should and do look to the brain for inspiration.

I'm taking issue with the rather ham-fisted series of papers that have come out in recent years aggressively pushing the agenda of connections between ML and neuro that just aren't there.

Are you sure that humans have done more net compute on Go than Deepmind just did? The Go game tree is _enormous_, humans are bias. We don't invent strategies from scratch, we use heuristics handed down to us from the pros (who in turn were handed down the heuristics from their mentors).

To me, it's not so interesting or surprising that the human initialized net performed worse. We just built the same biases and heuristics we have into that net.

Re:AlphaGo Zero: Learning from scratch by Houshalter on Wednesday October 18, 2017 @05:47PM (#15503511)
As far as we know the brain is just a "linear algebra blackbox". It's an uninteresting reduction since linear algebra can describe almost everything. Yes NNs aren't magic, but neither is the brain. Likely they use similar principles. Hinton has a theory about how real neurons might be implementing a variation of backpropagation and there are a number of other theories.
Re:AlphaGo Zero: Learning from scratch by ollin on Wednesday October 18, 2017 @11:58PM (#15505370)
>As far as we know the brain is just a "linear algebra blackbox"...Likely they use similar principles.

I'm not an expert, but my impression is that this is not really a reasonable claim, unless you're only considering very small function-like subsystems of the brain (e.g. visual cortex). Neural nets (of the nonrecurrent sort) are strict feed-forward function approximators, whereas the brain appears to be a big mess of feedback loops that is capable of (sloppily, and with much grumbling) modeling any algorithm you could want, and, importantly, adding small recursions/loops to the architecture as needed rather than a) unrolling them all into nonrecursive operations (like a feedforward net) or b) building them all into one central singly-nested loop (like an RNN).

The brain definitely seems to be using something backprop-like (in that it identifies pathways responsible for negative outcomes and penalizes them). But brains also seem to make efficiency improvements really aggressively (see: muscle memory, chunking, and other markers of proficiency), even in the absence of any external reward signal, which seems like something we don't really have a good analogue for in ANNs.

Re:AlphaGo Zero: Learning from scratch by Houshalter on Thursday October 19, 2017 @08:55AM (#15507447)
There are some parts of the brain we have no clue about. Episodic memory or our higher level ability to reason. But most of the brain is just low level pattern matching just like what NNs do.

The constraints you mention aren't deal breakers. We can make RNNs without maintaining a global state and fully unrolling the loop. See synthetic gradients for instance. NNs can do unsupervised learning as well, through things like autoencoders.

Re:AlphaGo Zero: Learning from scratch by fjsolwmv on Friday October 20, 2017 @03:39PM (#15518608)
A pattern matcher can learn high level reasoning. Reasoning is just a boolean circuit
Re:AlphaGo Zero: Learning from scratch by mannigfaltig on Wednesday October 18, 2017 @06:42PM (#15503919)
> It's an uninteresting reduction since linear algebra can describe almost everything.

The question is whether it can do so efficiently. As far as I know, alternating applications of affine transforms and non-linearities are not so useful for some computations that are known to occur in the brain such as routing, spatio-temporal clustering, frequency filtering, high-dimensional temporal states per neuron etc.

Re:AlphaGo Zero: Learning from scratch by andbberger on Wednesday October 18, 2017 @08:25PM (#15504487)
Hinton changes his opinion about what the brain is doing every 5 years... Hinton is not a neuroscientist...
Re:AlphaGo Zero: Learning from scratch by madez on Wednesday October 18, 2017 @08:46PM (#15504571)
If he changes his opinion, which I understand to be models of the brain in this case, and each iteration improves the model, then that is perfectly fine. It would be bad if someone did not change their view in case of inconsistent evidence.
Re:AlphaGo Zero: Learning from scratch by andbberger on Wednesday October 18, 2017 @08:50PM (#15504586)
For political opinions sure, but if he's changing his opinions so often ...

When you're a big scientific figure, I think that you have some extra responsibility to the public to only say things you're very confident about. Or otherwise very clearly communicate your uncertainty!!

Re:AlphaGo Zero: Learning from scratch by red75prime on Wednesday October 18, 2017 @11:24PM (#15505260)
> Hinton is not a neuroscientist...

It's not like neuroscientists know that either.

Re:AlphaGo Zero: Learning from scratch by fspeech on Wednesday October 18, 2017 @08:42PM (#15504559)
Agreed. If we announce that A* search is superhuman in finding best routes, most technorati would't bat an eye. Technically it is probably accurate to say that the results here show that neural networks can find good heuristics for MCTS search through unsupervised training in the game of Go. According to DeepMind authors:

"These search probabilities usually select much stronger moves than the raw move probabilities of the neural network; MCTS may therefore be viewed as a powerful policy improvement operator. Self-play with search – using the improved MCTS-based policy to select each move, then using the game winner as a sample of the value – may be viewed as a powerful policy evaluation operator. The main idea of our reinforcement learning algorithm is to use these search operators repeatedly in a policy iteration procedure ..."

The fact that this reinforcement training is unsupervised from the very beginning is quite exciting and may lead to better heuristics for other kinds of combinatorial optimization problems.

Re:AlphaGo Zero: Learning from scratch by etiam on Wednesday October 18, 2017 @04:20PM (#15502584)
Please don't refer to them as black boxes. The internals are fully observable.
Re:AlphaGo Zero: Learning from scratch by andbberger on Wednesday October 18, 2017 @08:46PM (#15504574)
Fully observable and we still have no idea what the hell it's doing.

Makes neuroscience seem kinda bleak doesn't it?

There has been a lot of great work lately building up a theory of how these things work, but it is very much still in the early stage. Jascha Sohl-Dickstein in particular has been doing some great work on this.

We don't even have answers to the most basic questions.

For instance (pedagogically), how the hell is it possible to train these things at all? They have ridiculously non-convex loss landscapes and we optimize in the dumbest conceivable way, first-order stochastic gradient descent. This should not work. But it does, all too often.

Not a great example because there are easy hand wavy arguments as to why it should work, but as far as proofs go...

The hand wavy argument goes as follows: - we're in like a 10000 dimensional space, for the stationary point we're at to be a true local minima that means each one of those 10000 dimensions goes uphill in either direction. It's overwhelming likely that there's at least one way out - there are many many different ways to set the params of the net for each function. Permutation is a simple example.

We really have no idea how these things work.

Anyone who tells you otherwise is lying to you...

Re:AlphaGo Zero: Learning from scratch by otabdeveloper1 on Friday October 20, 2017 @05:14AM (#15514486)
> Fully observable and we still have no idea what the hell it's doing.

Of course we do. It's matching a smooth multi-dimensional curve to sample data.

Re:AlphaGo Zero: Learning from scratch by fragsworth on Wednesday October 18, 2017 @04:27PM (#15502677)
It's a conceptual black box. There's no way for us to understand what each individual neuron is doing.
Re:AlphaGo Zero: Learning from scratch by blt on Wednesday October 18, 2017 @06:22PM (#15503769)
The tools we have developed so far are limited, but that's different from "there's no way". Many academics are working hard right now to better understand deep neural networks.
Re:AlphaGo Zero: Learning from scratch by etiam on Wednesday October 18, 2017 @04:39PM (#15502789)
Re:AlphaGo Zero: Learning from scratch by ionforce on Wednesday October 18, 2017 @04:28PM (#15502692)
Is there meaningful information in what one observes?
Re:AlphaGo Zero: Learning from scratch by aeleos on Wednesday October 18, 2017 @04:59PM (#15503024)
Yes, it turns out you can find meaningful information. etiam provided this https://arxiv.org/pdf/1312.6034.pdf The main issue is making sure what you are looking for is actually what the network is doing. You have to correctly interpret and visualize a jumble of numbers, which usually requires a hypothesis about how it worked in the first place. But assuming both go well you can gain meaningful information.
Re:AlphaGo Zero: Learning from scratch by fjsolwmv on Friday October 20, 2017 @03:49PM (#15518673)
Can I train an NN to visualize the numbers?
Re:AlphaGo Zero: Learning from scratch by autarch on Wednesday October 18, 2017 @06:42PM (#15503914)
> things like Chinese starts that are specific to certain cultures

While it's true that there are national styles of play, the Chinese opening is not called that because it's really popular among Chinese people. It's called that because a particular Chinese pro helped popularize it, even though it was invented by a Japanese amateur.

See https://en.wikipedia.org/wiki/Chinese_opening for some more info. FWIW, I (a caucasian American) use this opening all the time. It's just a generally good opening if you like a certain style of play.

Re:AlphaGo Zero: Learning from scratch by igravious on Thursday October 19, 2017 @05:27AM (#15506548)
> talk about things like Chinese starts that are specific to certain cultures

Came here to make this point.

It's Chinese Opening, not Chinese start – similarly recall that you have the French Defense / Italian Defense / Scandinavian Defense among chess opening variations and none of these implies that that opening variation is specific to that culture or nation.

Re:AlphaGo Zero: Learning from scratch by jasonwatkinspdx on Wednesday October 18, 2017 @04:07PM (#15502476)
> I wonder if this system produced more new styles of play.

One thing Alpha go has told us clearly is that it thinks human players over value the margin of victory vs the probability of victory.

Re:AlphaGo Zero: Learning from scratch by edanm on Wednesday October 18, 2017 @04:27PM (#15502676)
I'm not 100% sure I agree. It values probability of victory because that's it's goal. For humans, aiming only for probability of victory might not be as good, because we're much worse at estimating probabilities. So aiming for maintaining a large margin at all times is conceivably the best proxy that we can use in practice.
Re:AlphaGo Zero: Learning from scratch by pmontra on Wednesday October 18, 2017 @06:51PM (#15503975)
Agreed. I know I'm winning by 4 points but I have no idea about my probability of winning. However if I'm winning I know that I should play low risk moves and refrain from starting complicated fights. That increases the probability of winning. IMHO the exact value is out of reach for human beings.
Re:AlphaGo Zero: Learning from scratch by derefr on Thursday October 19, 2017 @03:47AM (#15506143)
It would be interesting to play human go, assisted by a go computer that doesn't say anything about moves, but rather just spits out, for each player, their current likelihood of victory if all further moves by both players were "what it would do."

That way, each player could know, at all times, (one major factor that goes into) their probability of winning. They'd still have to mentally adjust it for the likelihood of them and their opponent making an error, and how that can be controlled by making intimidating moves, etc. But it could lead to much tighter control on the abstract flow of the game.

It'd almost be like the computer was the general, issuing strategy, picking battles; and the human player the tactician, fighting those battles.

Re:AlphaGo Zero: Learning from scratch by pmontra on Thursday October 19, 2017 @10:41AM (#15508305)
There is computer go program (maybe Crazy Stone?) that analyzes a game record and annotates it with the winning percentage for every move.

Knowing that the opponent's winning probability changed from 52 to 57 was interesting only because it hints at a mistake. In case of such a large change the program suggests the move it would have played.

I saw an annotated game record and there were no variations: I remember a suggested move that made me wonder "why!?".

Another benefit of seeing the value of the winning probability is an assessment of who's ahead. However that's already possible with the score estimation that programs and go servers provide. Sometimes is crude, sometimes is good, but it's the score, not the winning probability that humans can estimate when playing. The best probability estimate I can make is: if the score is close and the game is still complicated, it's 50-50; if the score is close but the game is almost over, it's 95-5 for who's ahead. If the score is not close, the player with more points will probably win.

Re:AlphaGo Zero: Learning from scratch by fjsolwmv on Friday October 20, 2017 @04:53PM (#15519048)
Ha, like watching someone play a game and moaning or cheering at their plays.
Re:AlphaGo Zero: Learning from scratch by roenxi on Wednesday October 18, 2017 @05:38PM (#15503404)
The Go community learned to understand that the margin of victory is meaningless along time ago. The most famous game of Honinbo Dosaku, a famous Go player from the late 1600s, is arguably a game where he gave a handicap to his opponent and lost by one point. Lee Chang-Ho, who was the reigning champion in the late 90s, had a style that consistently tried to win by small margins.

AlphaGo now appears to be better than humans in all aspects of gameplay, and it better at calculating very thin margins of probability that a human cannot. This is not unique to any individual aspect of its gameplay; against humans it can also win by huge margins depending on what mistakes the human makes.

Re:AlphaGo Zero: Learning from scratch by hasenj on Wednesday October 18, 2017 @10:32PM (#15505062)
I think if AlphaGo foresaw it was losing by one point, it would start playing reckless moves, as it did against Lee-Sedol in the only match it lost against him.
Re:AlphaGo Zero: Learning from scratch by fjsolwmv on Friday October 20, 2017 @05:13PM (#15519141)
Risky, not reckless.
Re:AlphaGo Zero: Learning from scratch by hasenj on Monday October 23, 2017 @09:23PM (#15538459)
It wasn't risky. It was reckless. It played moves that made no sense and were obviously bad moves or pointless moves.

I'm talking specifically about game #4 of the Lee Sedol games.

Re:AlphaGo Zero: Learning from scratch by XR0CSWV3h3kZWg on Wednesday October 18, 2017 @04:27PM (#15502680)
Part of that could be trying to compensate for counting skills that aren't quite at machine levels.
Re:AlphaGo Zero: Learning from scratch by marcosscriven on Wednesday October 18, 2017 @03:12PM (#15502039)
Possibly a dumb q, but is ‘self play’ in any way related to ‘adversarial’ learning? I don’t see it mentioned in the article, but it reminds me of the principle.
Re:AlphaGo Zero: Learning from scratch by aeleos on Wednesday October 18, 2017 @03:36PM (#15502213)
In some ways it is, but the main difference is that adversarial learning (usually) produces a second neural network whose purpose is to exploit weakness is the first. Whereas reinforcement learning does not produce a second neural network to beat the first, it uses what it learned to solely improve the original.

As a side note, the main application I have seen with adversarial learning research is with photo recognition, but I guess you could have an adversarial network exist to help help improve an object recognition network. At that point it would probably become something between adversarial and reinforcement learning. However, with game based reinforcement learning, it doesn't require a second specific network as the adversary, it can easily just be paired against itself.

It isn't a dumb question, they are very similar in some ways. They mainly differ in what exactly the goal of the opponent is. In this case, it is to help improve itself, however in typical adversarial situations it is solely to exploit (become its adversary).