OpenAI ChatGPT: Optimizing Language Models for Dialogue

Re:OpenAI ChatGPT: Optimizing Language Models for Dialogue by gamegoblin on Wednesday November 30, 2022 @03:38PM (#33807301)

Embeddings are not for that. Embeddings take text and encode it into a high dimensional vector space. Similar texts will be closer together in the vector space.

The idea I was proposing was to use embeddings as a way to store and retrieve relevant "memories" so the AI could maintain coherence across time. I.e. whenever the user sends a message, we pull up the N most relevant memories (where relevance == closeness in the vector space) and include those in the prompt, so GPT3 can use the information when it forms its response.

[ Parent | on HN ]

Re:OpenAI ChatGPT: Optimizing Language Models for Dialogue by visarga on Thursday December 01, 2022 @04:32AM (#33814161)

I just implemented exactly this. In the corpus I put a few hundred papers I am interested in. Now I can ask a question, the search engine will find a few snippets and put them in the GPT-3 prompt.

[ Parent | on HN ]

Re:OpenAI ChatGPT: Optimizing Language Models for Dialogue by samching on Thursday December 01, 2022 @01:28PM (#33820389)

Any good guides for embedding generation?

[ Parent | on HN ]

Re:OpenAI ChatGPT: Optimizing Language Models for Dialogue by fareesh on Saturday December 03, 2022 @07:31AM (#33842982)

Yes this would be useful - does anyone have a crash course or something similar?

[ Parent | on HN ]