News for hackers. Stuff that matters.
Re:OpenAI Universe by BaronSamedi on Monday December 05, 2016 @09:53AM (#13105660)
Unless I missed something it looks like the AI has to learn from screen pixels instead of getting game state data. I don't like that approach at all. I understand that it's easy to implement for OpenAI but I think having the game developers provide a real bot-capable API is much better. I hope the latter is what Blizzard will provide for their DeepMind collaboration.
Re:OpenAI Universe by chongli on Monday December 05, 2016 @10:22AM (#13105856)
If your goal is to build General AI, getting access to the game state information is cheating and ultimately self defeating because it does not generalize across games. Expert humans are able to pick up and play a brand new game with a high degree of initial success despite zero knowledge of the memory states within the game. We do this by reliance on our vast knowledge of video game and literary tropes as well as experience playing past games. I have yet to see any video game bot make use of this stuff to figure out how to play Zelda, for example.
Re:OpenAI Universe by BaronSamedi on Monday December 05, 2016 @11:00AM (#13106264)
An AI which can learn any game from "looking" at a screen is a very ambitious goal. I doubt it is achievable in the near term. Humans can do it because of years of learning about the world. Personally, I prefer the more modest and achievable goal of teaching an AI to play complex real-time games such as Starcraft, LoL, and Dota--especially the latter 2 since they are team games.
Re:OpenAI Universe by chongli on Monday December 05, 2016 @12:24PM (#13107134)
Personally, I prefer the more modest and achievable goal of teaching an AI to play complex real-time games

Those sorts of goals are better suited to individual hobbyists. OpenAI is a blue sky research project set up by billionaires with the goal of improving all of mankind. I hope I'm not being too uncharitable when I say that your comment reminds me of those who scorned the Apollo program for its ambition.

Re:OpenAI Universe by BaronSamedi on Monday December 05, 2016 @07:41PM (#13111293)
I support research in AI of all kinds, however, its history is one of vastly over-promising and under-delivering. Modest but demonstrable progress is better, in my view, than yet more hype.
Re:OpenAI Universe by lawless123 on Monday December 05, 2016 @11:19AM (#13106452)
That's not their goal though.They want AGI.
Re:OpenAI Universe by mtgx on Monday December 05, 2016 @10:00AM (#13105703)
Seems unlikely. The focus seems to be on improving AI through "vision". The idea is to make the AI learn skills the same way a human would (at least in the first years of life). Google's AlphaGo also learned from screen pixels.

So these would be human-like bots, rather than bot-like bots, like you normally have in games. The bot would simply learn by doing, until it masters the game, not by getting access to game algorithms.

Re:OpenAI Universe by gwern on Monday December 05, 2016 @11:31AM (#13106573)
> Google's AlphaGo also learned from screen pixels.

It did not. It received the state of the board as one array, another board state array for capture/komi (since Go does have global state which is not visible purely from the board representation), and a few additional features to help it out with stuff like ladders. It was architected with convolutional layers, but over the Go grid, not pixels. See the AlphaGo paper pg11 for the exact structure of the input: http://www.postype.com/files/2016/04/08/16/05/03384c91046e8e...

Could it have learned from pixels (augmented by the additional necessary global state)? Sure. But that would've been a waste of computation since the visual layout of a Go board is fixed and static, unlike Atari games.

Re:OpenAI Universe by SamBam on Monday December 05, 2016 @10:03AM (#13105718)
> Google's AlphaGo also learned from screen pixels.

Source? That literally seems to make zero sense to me. Go can be represented in a super-simple state. Why make it spend millions of cycles learning to categorize pixels into that state you already have?

Re:OpenAI Universe by tiler on Monday December 05, 2016 @10:25AM (#13105882)
I would guess that they trained AlphaGo from many thousands of hours of match footage. Writing a computer vision script to segment / extract the data may cost cycles as you say, but would save many human hours by eliminating the need to re-watch the footage and literally type out state information for each move.
Re:OpenAI Universe by ironrabbit on Monday December 05, 2016 @02:08PM (#13108282)
Alphago was actually trained directly on game state (plus some extra computed state like "how many liberties will I have if I play this move" or "will I win this ladder"). A huge number of pro games (and countless amateur games) are available on servers like KGS in a nice computer-digestible format.
Re:OpenAI Universe by SamBam on Wednesday December 07, 2016 @12:35PM (#13124342)
Again, that doesn't make sense because the moves have almost certainly already been typed out by somebody. It's the same in chess. There are databases containing millions of games.
Re:OpenAI Universe by komaromy on Monday December 05, 2016 @03:22PM (#13109079)
I'm not too familiar with Go, but in chess we have millions of text-format games and very little real-time footage.
Re:OpenAI Universe by alkalait on Monday December 05, 2016 @10:01AM (#13105711)
Getting the game state data means deciding a-priori what features the AI should learn on. The whole point of the deep learning paradigm is to allow a machine to learn such features that enable good prediction, visualization, generation (aka. hallucination), etc.

Instead, researchers have provided the raw feed input data to these agents with the hope that the learned features could be interpreted as game state data by humans.

Re:OpenAI Universe by BaronSamedi on Monday December 05, 2016 @11:08AM (#13106337)
I would say that it is a point of deep learning rather than "the whole point". For an AI to interact with the real-world then building models from vision (as we do) makes a lot of sense. In the virtual world, however, it makes no sense. Model data is already available and the AI has no need for something as inefficient as vision. We humans have to use vision (and sound, etc) in games because we have do not have access to direct data feeds, computers have no such limitation. Why cripple the AI by imposing human limitations on it?
Re:OpenAI Universe by lawless123 on Monday December 05, 2016 @11:22AM (#13106485)
If they want their AGI to applicable to the real world or software with incomplete or insufficient APIS they have to do it the way they are doing it here.

There isn't an API for me to check if i'm still on the foot path and not the road as i walk down the street.

I can't use an API to tell me water is boiling and that i shouldn't stick my hand in it.

Re:OpenAI Universe by pakl on Monday December 05, 2016 @12:20PM (#13107092)
If the goal is to have AI that is aware of the real world (even a chatbot), then using game state is a crutch that doesn't help us solve the real problem.