In a sense, much of this work is similar to A* search and reinforcement learning. You have a limited view of the world, the responses to your actions are not directly causal, but if you can use the data you have to make reasoned decisions on what parts of the (extremely large) action space are unlikely to pay off, you can avoid wasting resources on them.