He’s talking about RLHF - reinforcement learning with human feedback (the process that trained ChatGPT), the training data for which is not publicly available.

And the point is that you don’t need to RLHF as long as you have access to another model that has been trained with RLHF that you can blackbox.