Re:Show HN: Improving RAG with chess Elo scores
by etk934 on Wednesday July 16, 2025 @02:57PM
(#44585667)
Will the reranker trained with MSE be better calibrated than those trained with InfoNCE? Will threshold on reranker scores be more useful in RAG applications?
[ Parent
| on HN ]
Re:Show HN: Improving RAG with chess Elo scores
by npip99 on Wednesday July 16, 2025 @03:40PM
(#44586023)
We tried a bradley-terry loss function, as calculated with https://hackmd.io/@-Gjw1zWMSH6lMPRlziQFEw/SJ8sRl1Zge
We found that MSE after elo-adjustment worked equally well. And, MSE lets you shuffle (q, d) across the dataset which has good statistical properties (Versus contrastive, which makes you sample the same query many times within a single minibatch)
In this case "InfoNCE" isn't applicable because the reranker's output is a scalar, not a vector. So that's why we checked both bradley-terry and MSE.