News for hackers. Stuff that matters.
Re:Vector database that can index 1B vectors in 48M by 1999-03-31 on Friday September 12, 2025 @03:38PM (#45225843)
1B vectors is nothing. You don’t need to index them. You can hold them in VRAM on a single node and run queries with perfect accuracy in milliseconds
Re:Vector database that can index 1B vectors in 48M by lyu07282 on Friday September 12, 2025 @04:08PM (#45226174)
Show your math lol
Re:Vector database that can index 1B vectors in 48M by Voloskaya on Friday September 12, 2025 @05:47PM (#45227152)
I assume by "node" OP meant something like a DGX node. Which yea, that would work, but not everyone (no one?) wants to buy a 500k system to do vector search.

B200 spec:

* 8TB/sec HBM bandwidth

* 10 PetaOPs assuming int8.

* 186GB of VRAM.

If we work with 512-dimensional int8 embeddings, then we need 512GB VRAM to hold them, so assuming we have 8xB200 node (~500k$++), we can easily hold them (125M vectors per GPU).

It takes about 1000 OPs to do the dot product between two vectors, so we need to do 1000*1B = 1TeraOPs, spread over 8 GPUs, that's 125 GigaOPs per GPU, so a fraction of a ms.

Now the bottleneck will be data movement between HBM -> chips, since we have 125M vectors per GPU, aka 64GB, we can move them in ~8 ms.

Here you go, the most expensive vector search in history, giving you the same performance as a regular CPU-based vectorDB for only 1000x the price.

Re:Vector database that can index 1B vectors in 48M by lyu07282 on Friday September 12, 2025 @11:38PM (#45229110)
Thanks for doing the math! I suppose if we are charitable in practice we would of course index and only offload partially to VRAM (FAISS does that with IVF/PQ and similar).
Re:Vector database that can index 1B vectors in 48M by adastra22 on Friday September 12, 2025 @05:35PM (#45227038)
1B x 4096 = 4T scalars.

That doesn't fit in anyone's video ram.

Re:Vector database that can index 1B vectors in 48M by kingstnap on Friday September 12, 2025 @09:07PM (#45228426)
Well we have AI GPUs now so you could do it.

Each MI325x has 256 GB of HBM. So you would need ~32 of em if it was 2 bytes per scalar.

Re:Vector database that can index 1B vectors in 48M by eknkc on Friday September 12, 2025 @05:37PM (#45227068)
I guess for 2D vectors that would work?

For 1024 dimensions even with 8 bit quantization you are looking at a terrabyte of data. Lets make it binary vectors, it is still 128GB of VRAM.

WAT?