News for hackers. Stuff that matters.

A comment on FANN: Vector Search in 200 Lines of Rust. Parent comment

Re:FANN: Vector Search in 200 Lines of Rust by huac on Thursday June 15, 2023 @05:21PM (#36346665)
I have gotten 10x speedups with SIMD on modern hardware - goes from fast to blazing. https://github.com/stillmatic/gollum/blob/main/vectorstore.g...

FWIW - Weaviate's pure Go implementation of cosine similarity is probably as fast as you can possibly get in pure go, without SIMD -- https://github.com/weaviate/weaviate/blob/2f44dbee52196656bd...

[ Parent | on HN ]
Re:FANN: Vector Search in 200 Lines of Rust by phillipcarter on Thursday June 15, 2023 @06:02PM (#36347258)
Yep, we'd go with what they did, except OpenAI embedding vectors are already normalized, so you can just do the dot product.
[ Parent | on HN ]
Re:FANN: Vector Search in 200 Lines of Rust by nearestn on Thursday June 15, 2023 @08:39PM (#36349259)
Weaviate actually does use SIMD for AMD64 with Go assembly (cosine distance is just normalized dot product) https://github.com/weaviate/weaviate/blob/master/adapters/re...
[ Parent | on HN ]
Re:FANN: Vector Search in 200 Lines of Rust by AA-BA-94-2A-56 on Thursday June 15, 2023 @08:50PM (#36349364)
Do you need anything special to run a Rust program with SIMD enabled?
[ Parent | on HN ]