Game replays (including all the fun data like player clicks, unit selection, movements, and exact timing) are stored in an easily parsable format. https://github.com/GraylinKim/sc2reader/wiki/Replay-Format-H...
Re:DeepMind Challenges for StarCraft
by HCIdivision17 on Monday March 21, 2016 @12:27PM
(#11329223)
This has the immense advantage that there's a large corpus of games available to learn from as well. I don't think Battle.Net saves all games ever (only locally on the computer), there's still many thousands available online.
But man, if Battle.Net did save all of them, there'd likely be hundreds of thousands, if not millions, available. That'd be pretty spectacular as a training set!
And you can run the StarCraft binary in a VM to be able to roll the game state back and forward.
Are there any VMs that give you memory-state level snapshots?
Xen does. Basically all of them have to, if you want to freeze your VM, or move it to a different server whilst it's running.
For their atari work, deepmind was using raw pixel data from the screen. You could do something similar for starcraft. I suspect the actual machine learning question is 'given a sequence of n frames of the game, where should i click next? repeat.'
A training set might look something like video of a game with a text file recording X,Y positions for every click and a timestamp that can be synced to the video. They need lots and lots of those.
From there things get drastically more complicated. To go from very unstructured data like that and form higher level representations that can be used for long term planning and strategy... That would be quite a feat.