Re:Adversarial policies beat professional-level Go AIs
by tasuki on Thursday November 03, 2022 @07:06PM
(#33459229)
Any human referee would declare KataGo the winner in any of these games, just like they did in the Robert Jasiek vs Csaba Mero case[0].
> But KataGo was trained under (modified) Tromp-Taylor which we also evaluate under, so we think from an AI security and robustness stand point it's interesting that it fails at the rule set it was trained on.
Have you done anything in particular to induce this, or is it just the case that KataGo is completely incompetent at evaluating what constitutes a pass-alive territory?
> That part is still a work in progress, we've only got it working against a victim without search for example
That would still be amazing. KataGo without search is still a beast.
---
By the way, your website is weird:
1. "strength of a top-100 European professional" - Europe doesn't have that many professionals?
2. "Yet our adversary achieves a 99% win rate against this victim by playing a counterintuitive strategy." - Well, if your main goal is to leave a stone in your opponent's territory, this strategy isn't very counterintuitive?
> But KataGo was trained under (modified) Tromp-Taylor which we also evaluate under, so we think from an AI security and robustness stand point it's interesting that it fails at the rule set it was trained on.
Have you done anything in particular to induce this, or is it just the case that KataGo is completely incompetent at evaluating what constitutes a pass-alive territory?
> That part is still a work in progress, we've only got it working against a victim without search for example
That would still be amazing. KataGo without search is still a beast.
---
By the way, your website is weird:
1. "strength of a top-100 European professional" - Europe doesn't have that many professionals?
2. "Yet our adversary achieves a 99% win rate against this victim by playing a counterintuitive strategy." - Well, if your main goal is to leave a stone in your opponent's territory, this strategy isn't very counterintuitive?
[0]: https://senseis.xmp.net/?DisputeMeroJasiek