Researchers are using synthetic data to train LLMs, especially for fine tuning, and especially instruct fine tuning. You are not up to date with recent work.
Neither was I.
> "synthetic data is bad“
I never said that… I said that it makes for poor training data, which it does.
> Researchers are using synthetic data to train LLMs, especially for fine tuning, and especially instruct fine tuning
Then those researchers are training with subpar datasets as the bias in that data will be compounded.
It’s a trade off since there’s only so much fresh data in form you want. If they could use entirely non synthetic data, I’m sure they would.
And again, you’re choosing to focus on this one point rather than my main point that prompt provide no moat.
> You are not up to date with recent work on LLMs.
There you go again making assumptions…
I think I’m done with this conversation though.
Training a model on synthetic data (obviously) increases bias present in the initial dataset[1], making for poor training data.
IIRC (this subject is a little fuzzy for me) using synthetic data for RLHF is equivalent to just using dpo, so if they did RLHF it probably wasn’t with synthetic data. They may have gone with dpo, though.
[1] https://arxiv.org/html/2403.07857v1