Viewing a single comment thread. View all comments

ThatSpysASpy t1_iupkljr wrote

The demonstrations shown in the paper are pretty unconvincing. In ordinary go scoring, dead stones are removed from the board at the end of the game, so the territory which supposedly isn't KataGo's would in fact be counted as its territory.

They say they use Tromp-Taylor rules, which requires all stones to be captured, but I would assume KataGo was trained with more standard human go rules. (Or at least they added some regularizer to make it pass once the value was high enough, otherwise humans playing vs it would get really annoyed).

19

ARGleave t1_iuq67g8 wrote

I replied to this in https://www.reddit.com/r/MachineLearning/comments/yjryrd/comment/iuq5hq9/?utm_source=reddit&utm_medium=web2x&context=3 but since this is currently the top-voted comment, I wanted to be clear that the scoring used is Tromp-Taylor, which KataGo was primarily trained with and which is the standard for evaluation in Computer Go.

Good point about the regularizer! KataGo does indeed have some functionality to encourage what it calls "friendly" passing to make it nicer for humans to play against, as well as some bonuses in favour of passing when the score is close. We disabled this and other such features in our evaluation. This does make the victim harder to exploit, but it's still possible.

I think it's reasonable to view this attack as a little contrived, but from a research perspective the interesting question is why it exists in the first place -- why didn't self-play discover this vulnerability and fix it during training? If self-play cannot be trusted to find it, then could there be more subtle issues.

26

picardythird t1_iuqhsya wrote

It is absolutely misleading to claim that Tromp-Taylor is "the standard for evaluation" in computer go.

Tromp-Taylor scoring has been used occasionally as a convenient means of simplifying the way that games are scored for the purposes of quantitative evaluation. However, area scoring (such as standard Chinese rules) or territory scoring (such as standard Japanese rules) are overwhelmingly more common, not to mention that these are actual rulesets used by actual go players.

Your claims are inflated and rely on overly-specific problem statements that do not map to normal (or even common) usage.

−7