I don't think it really matters whether he's talking set theory or category theory because the way he talks about set theory implies to me he'd have similar problems with category theory.

For example, he describes the result of Godel and Kohen, who together showed the continuum hypothesis is independent of ZFC, as being a failure/problem of ZFC. It's not. That you have significant theories that are independent of a given axiom system is a product of Godel's theorem. Modern mathematics operates with the assumption that most theories are true, false or unprovable.

But the situation also isn't specific to set theory. Any formal system is going to be subject to this "problem"

The thing is this state of affairs isn't appealing to the intuition. Godel himself was unhappy with his result. But it is basically inevitable if you get to the level of rigor of a formal system.