> We don’t believe that the courier change was responsible for the losses. We think we need more training, bugfixes, and to remove the last pieces of scripted logic in our model.
It is interesting to consider that manually scripted behavior may have been responsible for some of the more strange moments in the matchups; possibly they may have had hardcoded place-ward() routines that resulted in all of that clumped-together warding... or maybe even "check-roshan-pit()" without adequate training data to identify Roshan's respawning patterns?
The item purchasing is still scripted, which means that Five receives wards whether it wants them or not. One explanation for the ward dumping is that Five is just trying to free up inventory space.
Use the backpack? A hard five rarely owns 9 items and wards stack.
Roshan checking behavior evolved more naturally than the warding. At the start of training roshan was changed to have very low health so ai could stumble into the pit and be able to defeat roshan 1v1. Eventually the ai learned to check roshan and to group up to defeat it, but for some reason they never managed to learn about the respawn cooldown.
The cool thing is that bruteforcing computational power seems to get us decently close. I'm optimistic that with renewed interest in the reinforcement learning field, breakthroughs will be made on the algorithmic side within a matter of time.