News for hackers. Stuff that matters.
Re:Anyone got a contact at OpenAI. They have a spider problem by Eisenstein on Thursday April 11, 2024 @01:15PM (#40004508)
> Facebook alone probably has more data than the entire dataset GPT4 was trained on and it’s all behind closed doors.

Meta is happily training their own models with this data, so it isn't going to waste.

Re:Anyone got a contact at OpenAI. They have a spider problem by bevekspldnw on Thursday April 11, 2024 @02:23PM (#40005200)
Not Llama, they’ve been really clear about that. Especially with DMA cross-joining provisions and various privacy requirements it’s really hard for them, same for Google.

However, Microsoft has been flying under the radar. If they gave all Hotmail and O365 data to OpenAI I’d not be surprised in the slightest.

Re:Anyone got a contact at OpenAI. They have a spider problem by troq13 on Thursday April 11, 2024 @02:56PM (#40005545)
The company that made a honeypot VPN to access competitor's traffic? They are definitively keeping their hands off their internal data, yes.
Re:Anyone got a contact at OpenAI. They have a spider problem by Dr_Birdbrain on Thursday April 11, 2024 @05:58PM (#40007261)
I bet they are training their internal models on the data. Bet the real reason they are not training open source models on that data is because of fears of knowledge distillation, somebody else could distill LLaMa into other models. Once the data is in one AI, it can be in any AIs. This problem is of course exacerbated by open source models, but even closed models are not immune, as the Alpaca paper showed.