Re:Differential Transformer
by tucnak on Tuesday October 08, 2024 @09:24AM
(#41777110)
I'm led to believe this is mostly because "known unknowns" are not well-represented in the training datasets... I think, instead of bothering with refusals and enforcing a particular "voice" with excessive RL, they ought to focus more on identifying "gaps" in the datasets and feeding them back, perhaps they're already doing this with synthetic data / distillation.
[ Parent
| on HN ]
There are no comments on this comment yet.