Ah, that seems like an excellent system. It removes all of the guesswork of some formula-based solution. Empirical observations tend to be more accurate.
It might be worth hiring one of the top dota pros to coach OpenAI's bots: they could point out each small mistake that the bots made which pros wouldn't make. That might make it more tractable to beat the top team next year. (The actual dota coaches might be even better for this purpose than the pros, too.)
Looking forward to the outcome! The goal of "beat the top dota team" is one of those ambitious ones that few companies take on. It's really interesting to see the incremental progress.
How would the pro point out the mistakes to the bot? The bot cannot understand human language.
Not to the bot, but rather write down the mistakes and decide (as an AI programmer) how best to address them.
The solutions would probably take the form of adjusting weights, adding additional dimensions, removing or refactoring existing signals, or deciding that extra training would naturally solve the problem.
We don't claim this is perfect, which is why we label the chart "estimated MMR".