o1's thought chains aren't traditional shoggoth mask RLHF/DPO/what have you, the reinforcement metric is the scores discussed in the video.