https://arstechnica.com/information-technology/2022/11/new-go-playing-trick-defeats-world-class-go-ai-but-loses-to-human-amateurs/
Adversarial policy attacks blind spots in the AI—with broader implications than games.
Benj Edwards - 11/7/2022, 12:43 PM
...
Last week, a group of AI researchers published a paper outlining a
method to defeat KataGo by using adversarial techniques that take
advantage of KataGo's blind spots. By playing unexpected moves
outside of KataGo's training set, a much weaker adversarial
Go-playing program (that amateur humans can defeat) can trick
KataGo into losing.
...
In this case, the researchers' policy uses a mixture of a neural network
and a tree-search method (called Monte-Carlo Tree Search) to find
Go moves.
KataGo's world-class AI learned Go by playing millions of games against
itself. But that still isn't enough experience to cover every possible scenario, which leaves room for vulnerabilities from unexpected behavior. "KataGo generalizes well to many novel strategies, but it does get weaker
the further away it gets from the games it saw during training," says
Gleave. "Our adversary has discovered one such 'off-distribution' strategy
that KataGo is particularly vulnerable to, but there are likely many others."
Gleave explains that, during a Go match, the adversarial policy works by
first staking claim to a small corner of the board. He provided a link to an example in which the adversary, controlling the black stones, plays largely
in the top-right of the board. The adversary allows KataGo (playing white)
to lay claim to the rest of the board, while the adversary plays a few easy-to-capture stones in that territory.
"This tricks KataGo into thinking it's already won," Gleave says, "since its territory (bottom-left) is much larger than the adversary's. But the bottom-left territory doesn't actually contribute to its score (only the
white stones it has played) because of the presence of black stones there, meaning it's not fully secured."
As a result of its overconfidence in a win—assuming it will win if the game ends and the points are tallied—KataGo plays a pass move, allowing the adversary to intentionally pass as well, ending the game. (Two consecutive passes end the game in Go.) After that, a point tally begins. As the paper explains, "The adversary gets points for its corner territory (devoid of
victim
stones) whereas the victim [KataGo] does not receive points for its unsecured territory because of the presence of the adversary's stones."
--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)