The idea is to see if the bot is playing as smart as a human. To measure this, you must factor out any advantages not available to humans. This is valuable feedback when developing an intelligent agent, but that doesn't mean you can't remove those artificial handicaps once going to production with something like self-driving cars.

Edit:

Alphastar does already factor out reaction time:

“We measured how quickly it reacts to things,” Silver said. “If you measure the time between when AlphaStar perceives the game. From when it observes what’s going on, then has to process it, and then communicate what it chooses back to the game. That time is actually closer to 350ms. That’s on the slow side of human players.” https://venturebeat.com/2019/01/24/alphastar-deepmind-beats-...