FANN: Vector Search in 200 Lines of Rust

Re:FANN: Vector Search in 200 Lines of Rust by zainhoda on Thursday June 15, 2023 @04:35PM (#36346152)

Same — I just asked ChatGPT to generate Go code to do cosine similarity and I was surprised when the code it generated worked and the worst performance I get is like 10ms

[ Parent | on HN ]

Re:FANN: Vector Search in 200 Lines of Rust by phillipcarter on Thursday June 15, 2023 @04:38PM (#36346196)

Yep. If you parallelize with goroutines it'll be faster, but we're talking about shaving a few ms. The effectiveness of a for loop crunching floats in a single thread is not to be underestimated.

[ Parent | on HN ]

Re:FANN: Vector Search in 200 Lines of Rust by huac on Thursday June 15, 2023 @05:21PM (#36346665)

I have gotten 10x speedups with SIMD on modern hardware - goes from fast to blazing. https://github.com/stillmatic/gollum/blob/main/vectorstore.g...

FWIW - Weaviate's pure Go implementation of cosine similarity is probably as fast as you can possibly get in pure go, without SIMD -- https://github.com/weaviate/weaviate/blob/2f44dbee52196656bd...

[ Parent | on HN ]

Re:FANN: Vector Search in 200 Lines of Rust by phillipcarter on Thursday June 15, 2023 @06:02PM (#36347258)

Yep, we'd go with what they did, except OpenAI embedding vectors are already normalized, so you can just do the dot product.

[ Parent | on HN ]

Re:FANN: Vector Search in 200 Lines of Rust by nearestn on Thursday June 15, 2023 @08:39PM (#36349259)

Weaviate actually does use SIMD for AMD64 with Go assembly (cosine distance is just normalized dot product) https://github.com/weaviate/weaviate/blob/master/adapters/re...

[ Parent | on HN ]

Re:FANN: Vector Search in 200 Lines of Rust by AA-BA-94-2A-56 on Thursday June 15, 2023 @08:50PM (#36349364)

Do you need anything special to run a Rust program with SIMD enabled?

[ Parent | on HN ]