For that scale FAISS might be not enough, as it keeps the data in memory and is simply hard to scale. It's worth considering to use a proper vector database in a distributed mode to spread the load, or just allow storing some data on disk: https://qdrant.tech/articles/memory-consumption/