Not really. If they are ingesting peoples data to train their model, at what point are they checkpointing? Is data I submit 2 months ago that I would want removed not present in a recent checkpoint or would they have to "unwind" the last 2 months of training to remove it? What about people that would want something removed 6 months prior? What if something in the core model is to be removed?

Checkpointing may help, sure, but it isnt going to allow you to remove a single piece of training data without headache, and potential substantial retraining.

If that 3% from the GPs comment comment is uniformly distributed throughout the training history, the only way it can be reliably removed is to retrain from scratch.

> If they are ingesting peoples data to train their model, at what point are they checkpointing?

Multiple times per day if they're not incompetent. And I don't think they're incompetent.

Are you implying that removing the diff between checkpoints achieves the same effect? Ive never heard of this, but I suppose it may be possible.

I suppose the "ghost" of the removed weights would also have shaped subsequent training though...

Interesting idea...