News for hackers. Stuff that matters.

A comment on What Is a Vector Database. Parent comment

Re:What Is a Vector Database by sidi on Friday May 05, 2023 @08:17AM (#35828267)
There currently isn't a way to filter docs alongside a KNN query, and the dimension support is limited to 1024 (a Lucene limitation) and OpenAI embeddings are 1536 dimensions - also indexing performance is not comparable. Wishing this changes, as they're a good stack for the reasons you state
[ Parent | on HN ]
Re:What Is a Vector Database by softwaredoug on Friday May 05, 2023 @08:20AM (#35828292)
True though I do think 2k dims is coming it 8.8
[ Parent | on HN ]
Re:What Is a Vector Database by peterstjohn on Friday May 05, 2023 @11:56AM (#35830997)
Are they forking Lucene or somehow getting the Lucene devs to increase that limit? Because this PR has been open for over a year now: https://github.com/apache/lucene/issues/11507
[ Parent | on HN ]
Re:What Is a Vector Database by softwaredoug on Friday May 05, 2023 @02:01PM (#35832911)
No - they just did something in Elasticsearch to make their own FieldType https://github.com/elastic/elasticsearch/pull/95257
[ Parent | on HN ]