We use sim2real with distillation-based transfer :)
This means no need to do the rollouts on real robots first! Please check out https://sgs-rl.github.io for some transfer examples (albeit a simpler setting then the multi-part NIST board we examine in AlphaNIST).
(This whole thing is awesome. I think they will need to get the number of rollouts needed down before it translates well to real robots — running hundreds of thousands of rollouts on small parts with a real rollout will be a real logistical mess when cleaning up parts on the floor and dealing with everything that the robots physically break while learning.)