Game theory states that exploitability(s) is basically utility(s, best_response(s)). Suspect you aren't taking his claim about exploitability seriously or are using a different definition than the one mathematicians often use. It is a much stronger claim than you seem to realize and it is not mathematically rigorous or particularly well informed by the equations to claim that outcomes scaled by probabilities are off topic.

Might help to talk about decision theory rather than game theory. Check this:

P(heads=0.6), reward = 1. Notice that an agent optimizing expected value will recommend always heads for every estimate of P(heads) > 0.5. Here the observed probabilities can be 0.6 and the estimate could be 0.9 and even though there is error in the estimate of expected value it doesn't translate to a difference in expected value of the recommended strategy given that estimate. This is worth dwelling on with regard to his argument, because it helps to notice that his argument for why exploitation exists isn't even valid for arguing that error expected utility in simpler decisions problems exists. If he doesn't even have good argumentative structure for a much much less complicated decision problem, why is introducing exploitability, which is done with respect to multiple agents, going to be correct?

Probably still exploitable, but his argument for why is trash and he doesn't at all establish that it is particularly exploitable relative to other situations.

That is why I can give an example of a superhuman agent that always errors in this way, but isn't exploitable. His argument isn't valid.