For anyone curious to see how to build one of these vector database chat models in action I built one from (semi scratch) in a colab environment and inference with llama 2 on my live stream last week https://www.youtube.com/live/kBB1A2ot-Bw?feature=share

Big challenge with this set up is doing the semantic similarity search at scale. Pinecone has some good docs on their data structures for scaling large vector databases