News for hackers. Stuff that matters.
Re:Show HN: Turning books into chatbots with GPT-3 by leobg on Tuesday January 24, 2023 @04:16PM (#34510538)
You don’t. You cut it into snippets. For those you create embeddings which allow you to rank them by semantic similarity to a query. You then prompt GPT3 with the question plus, say, the three most relevant snippets from the book.

The most difficult thing about the process is preventing the model from making stuff up.

Re:Show HN: Turning books into chatbots with GPT-3 by kreas on Tuesday January 24, 2023 @04:53PM (#34511033)
This is exactly what I'm working on! My project is taking Zoom conversation, using pyannote for speaker diarisation, whisper for transcription, pinecone.io for semantic search, then feeding that into GPT-3 so we can ask questions about conversation.

For us this is super useful because it's not unusual for our discover sessions to last days and we're all terrible at taking notes.

As a nerd, my brain is already buzzing on ways that I could use this for my groups D&D campaigns.

Re:Show HN: Turning books into chatbots with GPT-3 by mistermann on Tuesday January 24, 2023 @05:29PM (#34511497)
This sounds so interesting, do you have any plans to write up a more detailed description of it??
Re:Show HN: Turning books into chatbots with GPT-3 by kreas on Tuesday January 24, 2023 @05:55PM (#34511812)
I've got tons of notes so it shouldn't be too hard to do a write up. Currently it's in a private repo, but if I can get sign-off from my boss I'll open source it.
Re:Show HN: Turning books into chatbots with GPT-3 by mistermann on Sunday January 29, 2023 @02:48PM (#34571946)
Any idea when you might find out??? :)
Re:Show HN: Turning books into chatbots with GPT-3 by sputknick on Tuesday January 24, 2023 @06:33PM (#34512185)
Are you getting good results when summarizing a human speaking? On my project, even though Whisper does a good job translating it, I'm not happy with the query results. My theory is that GPT-3 is designed for written word and the way people speak and the way they write are structurally different. Or I'm just figuring this out and I'm not good enough at it yet.
Re:Show HN: Turning books into chatbots with GPT-3 by leobg on Wednesday January 25, 2023 @04:33PM (#34524485)
It’s often not enough to just index the snippets themselves. You may need to augment them. For instance, you may need to keep track of the context, and prepend it to the actual snippet that you want to index.

The important thing in such a pipeline is not GPT 3. The important thing is the retrieving/ranking algorithm that finds the most relevant snippets and feeds them into GPT 3. The latter is only the mouthpiece, if you will.

In fact, you might even find that you’re better off without it (no confabulation, ground truth data).

Re:Show HN: Turning books into chatbots with GPT-3 by batch12 on Tuesday January 24, 2023 @08:48PM (#34513336)
Interesting, I have been playing with something similar to use as a knowledge lookup tool stapled to a dynamic prompt builder.
Re:Show HN: Turning books into chatbots with GPT-3 by d4rkp4ttern on Wednesday January 25, 2023 @07:11AM (#34517314)
How do you create the embeddings? Is there a GPT3 API that returns a paragraph’s embedding vector?
Re:Show HN: Turning books into chatbots with GPT-3 by leobg on Wednesday January 25, 2023 @04:24PM (#34524417)
There is. But I’m not sure how good they are. Nils Reimer wrote a Medium post about them, concluding that they performed worse than SOTA sentence transformer models.