kerbal

Same impression here. Rolling my own to learn more in the process.

What I really don’t like is the fact that the new chat endpoint doesn’t have the logprobs option.

For InstructGPT models you can view token probability but for newer models you cannot - another thing that “Open”AI decided that we shouldn’t know.

Oh, yes - in my tests it worked better most of the time but there were some cases where the results were worse. Regarding the “pyramids” - I think it might work better with actual questions.

Exactly. The fake (hypothetical) answer is usually longer than the question and often contains words that match the real answer even when the domain of question and answer is different ie “how to get out of jail?” asked in the context of monopoly game and the hypothetical answer is in the context of a real jail. It sounds stupid but it works and is super easy to implement.

I’ve also used similar approach to build Q&A system for PDF files (my main use case - board game manuals). OpenAIs embeddings are nice to play with. Also there is an easy technique for getting better results with dense search - Hypothetical Document Embeddings (https://arxiv.org/abs/2212.10496).