On your second point about efficient filtering, check out this article I wrote outlining how filtered vector search works in Weaviate: https://towardsdatascience.com/effects-of-filtered-hnsw-sear...
For even more details on filtering, check the documentation: https://www.semi.technology/developers/weaviate/current/arch...
Anyway, for me, the number one priority was latency and it is hard to beat on-premise search for that.
Even then, a vector search API is just one component you will need in your stack. You need to pick the right model, create vectors (GPU intensive), then possibly combine search results with keyword based search (say BM25) to improve accuracy etc. I am still waiting to see an end-to-end API doing all this.
Vespa.ai supports combining dense vector search with keyword search and ranking, see https://docs.google.com/presentation/d/1vWKhSvFH-4MFcs4aNa9C...
There is also a Vespa sample application (open source, Apache 2) demonstrating multiple different retrieval and ranking strategies over at https://github.com/vespa-engine/sample-apps/blob/master/msma...
That’s kinda the idea of Weaviate. You might like the Wikipedia demo dataset that contains all this. You indeed need to run this demo on your own infra but the whole setup (from vector DB to ML models) is containerized https://github.com/semi-technologies/semantic-search-through...
It's an interesting point tho. Maybe it's good to add this to the docs as well
And even if you do just want ANN and nothing else, some people just want to make API calls to a live service and not worry about anything else.
And you’re right, it wasn’t easy to build.
1. Incrementally updating the search space. Not that easy to do, and becomes more important to not just do the dumb thing of retraining the entire index on every update for larger datasets.
2. Combining vector search and some database-like search in an efficient manner. I don't know if this Google post really solves that problem or if they just do the vector lookup followed by a parallelized linear scan, but this is still an open research/unsolved problem.