One of the example possibilities mentioned in one of his essays or interviews was, what if with some poorly-understood tweaks to neural network training, it started working much better at scale? At the time I thought he was really reaching with that one.