But I would love to see more informed insight/discussion on this.
https://mastodon.social/@tristanbuckmaster/11723647135247030...
OpenAI explicitly uses user feedback (the thumbs up or thumbs down ratings), as RLHF to train models. However, this feedback is anonymized and stripped of user identifiers. If Buckmaster ever used this feature, then that conversation would be anonymized, saved, and used for training, but not tied back to him.
They cannot issue a blanket denial (which people so desperately desire), and instead repeat that "it's very unlikely" (which pisses people off), because they cannot in good faith claim to have zero data at all.
the real tricky question would be: "to what extent does a year of work, put into Codex, towards an eventual proved forced blowup for the Euler equations go into training data and become an influence on agents told to attack NS?", and given that the first signal that OAI got to assign all firepower to NS rather than broadly spray all Millennium Prize + other high profile questions was a proven unforced blowup of the Euler equations after only 50 hours, I would say this extent seems likely to be decently high
If you think these companies are not training on your prompts you are incredibly naive. These models were built by stealing and pirating literally everything they can get their hands on no matter the legality. AI companies are always very specific about what they're not doing - in a way that you can drive a truck through the loopholes