Second, what you are suggesting is probably best done as a secondary process outside of learning the Nash equilibrium. If you knew an opponent's policy, you would need to recalculate your optimal counterplay for that specific policy. This is completely orthogonal to the goal of this paper which is to learn the Nash equilibrium through self-play alone.
See the Deep Mind "Player of Games" paper from last year for an agent that takes a more game theoretic approach, which is probably needed for "simpler" games like Poker, that we can play to higher levels of accuracy: https://arxiv.org/pdf/2112.03178.pdf
What is the SOTA on solving non-adversarial (single player?) POMDPs? Are those considered to be much simpler problems?
In case you are wondering about cooperative multi-agent games, I would check this group's publications: https://www.cs.ox.ac.uk/people/publications/date/Shimon.Whit...
Well, solving a Partially Observed Markov Decision Processes in general isn't just NP-complete but actually undecidable. So I'm not sure how one measures SOTA (state of the art).
You measure it by practical performance on a test suite of important POMDPs and games.
POMDP just means "observations are not state" and that you need to use a stateful policy to infer the state somehow, but without further assumptions it's difficult to answer this question