Re:QwQ-32B: Embracing the Power of Reinforcement Learning
by GTP on Thursday March 06, 2025 @07:02AM
(#43279238)
Sorry, could you please explain what this means? I'm not into machine learning, so I don't get the jargon.
Re:QwQ-32B: Embracing the Power of Reinforcement Learning
by smallerize on Thursday March 06, 2025 @06:30PM
(#43286087)
Well I can't be positive, but it looks like some of the factors that support a long context length might be set wrong. https://blog.eleuther.ai/yarn/
Presently, vLLM only supports static YARN, which means the scaling factor remains constant regardless of input length, potentially impacting performance on shorter texts. We advise adding the rope_scaling configuration only when processing long contexts is required.