Re:Show HN: Factual AI Q&A – Answers based on Huberman Lab t...
by jamesbriggs on Sunday December 18, 2022 @08:35AM
(#34037499)
I should add, Riley used the ada embedding model (rather than sentence transformers). Performance wise they should be similar (in ability to encode meaning accurately) but the ada model can encode a much larger chunk of text. I don't know exact numbers but something like 1-2 pages of text in a typical corporate PDF. Whereas sentence transformers are typically limited to around a paragraph of text.
Typically you'd split the text in paragraph sized chunks to handle this requirement of sentence transformers, with GPT-3 embeddings you naturally have more flexibility there
The way I built it is documented here: https://www.pinecone.io/learn/openai-whisper/
Afaik it's the same approach as Riley, that is:
- Scrape audio of YouTube videos
- Transcribe to text with OpenAI's Whisper
- Use sentence transformer to create embeddings of text
- Index embeddings (with transcribed text, timestamps, and video URL attached) in Pinecone's vector database
- Wrap up the querying functionality in a nice UI
(this is for the search functionality)
If wanting to replicate the Q&A part, I also built something similar and wrote about it (https://youtu.be/coaaSxys5so) - it's essentially the same process but we return text snippets to GPT-3 along with the original question and it generates an answer