News for hackers. Stuff that matters.
Re: by loxias on Thursday August 17, 2023 @06:05AM (#37159473)
> it is really problem dependent as is throughput. Recall and latency numbers reported in benchmarks are typically on very well curated and structured datasets and average across all queries

This is correct. :) Don't worry, I know enough to not trust any published benchmarks on this topic... (I'm also not your target market. I wrote my first "vector DB" in 2001 for music recognition.)

I still think it's crucial to include just a few more facts though, because otherwise the statement is meaningless.

Consider:

A. "we can find an approximate NN match, euclidean, D=768, N=70000000, under 100ms on a modern laptop"

vs

B. "we can find an approximate NN match, euclidean, D=2, N=70000000, under 100ms on a modern laptop"

vs

C. "we can find an approximate NN match, euclidean, D=768, N=70000000, under 100ms on 1000x modern laptops"

Notice how B and C aren't impressive, they're trivially beatable. :)

There are no comments on this comment yet.