Go player here. Even for human-vs-human Go, fast games also widen the gap in favor of the stronger player. Although in this case the mechanism might be somewhat different - stronger players are more experienced, and their "intuitive" skills (the subconscious neural networks that tell you in a second which parts of the board look interesting) benefit from more training.

But perhaps the same reason applies to human-vs-AI Go. AlphaGo's architecture bears a striking resemblance to how the human mind operates when playing Go.