News for hackers. Stuff that matters.
Re:Differential Transformer by tucnak on Tuesday October 08, 2024 @09:24AM (#41777110)
I'm led to believe this is mostly because "known unknowns" are not well-represented in the training datasets... I think, instead of bothering with refusals and enforcing a particular "voice" with excessive RL, they ought to focus more on identifying "gaps" in the datasets and feeding them back, perhaps they're already doing this with synthetic data / distillation.

There are no comments on this comment yet.