Re:ANN v3: 200ms p99 query latency over 100B vectors
by benesch on Sunday January 25, 2026 @03:59PM
(#46758156)
Yep, we're well aware of the selection bias effects in product feedback. As we grow we're thinking about how to make our product more accessible to small orgs / hobby projects. Introducing a local dev environment may be part of that.
Note that we already have a in-your-own-VPC offering for large orgs with strict security/privacy/regulatory controls.
Re:ANN v3: 200ms p99 query latency over 100B vectors
by nik9000 on Sunday January 25, 2026 @03:44PM
(#46757996)
As someone who works for a competitor, they are probably right holding off on that segment for a while. Supporting both cloud and local deployments is somewhere between 20% harder and 300% harder depending on the day.
I'm watching them with excitement. We all learn from each other. There's so much to do.
Re:ANN v3: 200ms p99 query latency over 100B vectors
by sa-code on Sunday January 25, 2026 @03:31PM
(#46757853)
Can confirm. With a setup that works offline, one can
- start small on a laptop. Going through procurement at companies is a pain
- test things in CI reliably. Outages don’t break builds
- transition from laptop scale to web scale easily with the same API with just a different backend
Otherwise it’s really hard to justify not using S3 vectors here
The current dev experience is to start with faiss for PoCs, move to pgvector and then something heavy duty like one of the Lucene wrappers.
It's only occasional because the people who care about dev environments that work offline are most likely to just skip you and move on.
For actual developer experience, as well as a number of use cases like customers with security and privacy concerns, being able to host locally is essential.
Fair enough if you don't care about those segments of the market, but don't confuse a small number of people asking about it with a small number of people wanting it.