Re:You Should Probably Pay Attention to Tokenizers
by NitpickLawyer on Wednesday October 23, 2024 @06:26PM
(#41929827)
IIUC they're doing some sort of "q/a" for each chunk from documents, where they ask an LLM to "play the user role and ask a question that would be answered by this chunk". They then embed those questions, and match live user queries with those questions first, then maybe re-rank on the document chunks retrieved.
[ Parent
| on HN ]
Re:You Should Probably Pay Attention to Tokenizers
by nullc on Thursday October 24, 2024 @04:01AM
(#41933189)
I wonder if backwards a backward predicting LLM might do better.
E.g. augment the data with "Certainly, the document matching your question is <document>" and then sample the backwards completion to get candidate questions.