News for hackers. Stuff that matters.
Re:OpenAI bots competing against Humans right now by ufo on Sunday August 05, 2018 @07:32PM (#17694151)
In the last game of the series the bots were forced to play an intentionally terrible hero lineup. The humans finally got to win a game but more interesting for me was that the times when the AI did seemingly crazy things were much more common.

I wonder if this is an artifact of the training methodology: maybe if your team is very weak then your choices are also weaker, and reinforcement learning doesn't work as well?

Re:OpenAI bots competing against Humans right now by tux3 on Sunday August 05, 2018 @07:40PM (#17694181)
It reminds me of the Go AIs going on tilt when they're far behind.

When the win percentage for Go AIs gets to around 5%, every action it can take results in a losing game so it can't make the difference between normal play and super strange moves anymore.

When every choice is really bad, humans tend to still go with their normal strategy and wait for their chance to turn things around, but bots assume the opponent is playing perfectly, so they act like their winrate is going to stay near zero no matter what they do.

Re:OpenAI bots competing against Humans right now by nopinsight on Monday August 06, 2018 @11:42AM (#17698602)
If this is true (which it might be, I haven’t studied these systems in detail), then an obvious fix is to get the AI to randomly train with ‘weaker’ versions of itself (perhaps while the strong instance is handicapped) in addition to the latest generation.

Several levels of weak opponents should be used, with varying probabilities, to tune the AI’s robustness against real-world, imperfect competitors.

Re:OpenAI bots competing against Humans right now by hackandtrip on Sunday August 05, 2018 @07:58PM (#17694245)
I'd say that those were crazy things, but not too much. AI understood that teamfight were impossibile to win, and so the really weird choices (sven dying for the tower lol) . I'd say that matches with such lineup are not too much and so they didn't really know, but they did an awesome job and kept things balanced for so long
Re:OpenAI bots competing against Humans right now by Rychard on Sunday August 05, 2018 @08:42PM (#17694405)
> really weird choices (sven dying for the tower lol)

Taking a tier 2 tower nets everyone on the team 120 gold (a further 150+ gold goes to the hero who gets the last hit), and losing Sven probably gave the opposing team less than was gained.

Perhaps the AI simply placed more value on increasing the total net worth of the team than it valued saving the life of one of its core heroes. Additionally, there was no guarantee that he would have been able to escape, as Sven was deep on the enemy's side of the map, and there's a very real possibility that he could have been ganked from someone in the jungle had he attempted to retreat.

Re:OpenAI bots competing against Humans right now by drusepth on Monday August 06, 2018 @02:43PM (#17700340)
Dota player here: other metrics that could have been involved in the trading decision are:

- Potential chance the enemy team would deny the tower before another friendly hero could take it (netting Sven's team 0 gold for the time spent whacking away at it)

- Map vision (removing a T2 often cuts a significant section of map awareness away, since the tower is no longer providing vision or protection)

- XP gains (Sven won't gain any XP while dead, nor from killing the tower)

- Creep equilibrium (this is less important, or at least thought about less often, later on in the game and past T1 towers, but might've been a factor in drawing the creep clash point to a particular location)

- Dictating team net worth averages (to some extent, if they predicted a loss in opponents forcing a teamfight or predicted a likely pickoff, gold lost could be minimized now by taking a death early, lowering the average net worth on the team).

Obviously, there are others and these can also be mixed and matched in various ways (e.g. cutting off map vision so they can more safely farm additional jungle creeps).

Not saying any of these aspects _were_ a part of the decision to trade Sven for a tower, but.. just wanted to include a few more subsurface aspects that _could_ be used in such a decision.

Re:OpenAI bots competing against Humans right now by dooglius on Sunday August 05, 2018 @07:48PM (#17694213)
That's probably part of it, but another explanation would be that the AI wouldn't choose such a lineup on its own, and so it doesn't train on the relevant gameplay.
Re:OpenAI bots competing against Humans right now by zaarn on Monday August 06, 2018 @03:47AM (#17695829)
I think this might be a result of "the only winning move is not to play", so to speak. If the game is, in the mind of the AI agents, unwinnable, not playing is not an option, therefore it begins to pick random actions instead.

I'm not sure if the AI can surrender (I only managed to watch the first two games as it was rather late at night) but it might be a path to explore; having the AI give up if the game cannot be won anymore.

Re:OpenAI bots competing against Humans right now by wwhitlow on Monday August 06, 2018 @02:11PM (#17700053)
At what percentage would you allow an AI to consider a game unwinnable? While an AI that behaves erratically when the odds are low might be worth allowing it to be considered forfeit worthy, but the thing about humans is we make mistakes. Therefore an ideal AI that can continue to execute reasonable moves should have a lower percentage threshold where it decides to forfeit. See this match[0] for an example of a spectacular comeback that I feel an AI might have considered forfeit worthy if not well defined.

[0]: https://youtu.be/LwSQv_sNZBI

Re:OpenAI bots competing against Humans right now by ionforce on Monday August 06, 2018 @02:29PM (#17700199)
Maybe this is a limitation of self-play. If the opponent an AI faces during training is always optimal, then there's no surface area of mistakes. The losing AI, in its model/mind, knows that the game is over after a specific threshold. So it hasn't learned how to optimize for capitalizing on mistakes.

I wonder if this situation can be fixed by adding more randomness. For example, force AI'1 to be in a losing position to AI'2, but then suddenly switch the power level of AI'2 to be much weaker (where mistakes happen) so that AI'1 learns how to fight its way out of tough situations.

Re:OpenAI bots competing against Humans right now by ufo on Monday August 06, 2018 @06:09PM (#17702209)
One of the most interesting takeaways from the post game interview for me was that the AI can be very stupid if you just blindly throw it in a self-play setting but with clever use of randomization (modifying power levels) and action restrictions (for example, only allowing the agent to spend an anti-invis item when a nearby enemy goes out of sight) it is possible to provide better learning opportunities for the AI.
Re:OpenAI bots competing against Humans right now by ionforce on Monday August 06, 2018 @06:13PM (#17702226)
> for example, only allowing anti-invis items when an enemy goes out of sight

These are the kind of actions you specifically don't want to code in because you're throwing in human knowledge. You want the AI to learn by itself that using anti-invis when everyone is visible is a low-value move.

The purist in me was even mad that they had a hand-crafted evaluation function. (e.g. prefer gold, prefer taking towers, each given some arbitrary value)

Re:OpenAI bots competing against Humans right now by make3 on Sunday August 05, 2018 @10:31PM (#17694766)
maybe the team never picked lineup it knew were bad, and as such never trained with them