Re:Vectors are over, hashes are the future
by packetlost on Friday October 07, 2022 @04:54PM
(#33126742)
PineCone seems interesting. Is the storage backend open source? I've been working on a persistent hashmap database that's somewhat similar (albeit not done) that should have less RAM requirements than bitcask (ie. larger than RAM keysets)
Although we may open-source parts in the future, currently no part of Pinecone is open-sourced. Instead, there are several proprietary index types available, packaged along with hardware/compute resources into what we call “pods.”
The demand for vector embedding models (like those released by OpenAI, Cohere, HuggingFace, etc) and vector databases (like https://pinecone.io -- disclosure: I work there) has only grown since then. The market has decided that vectors are not, in fact, over.