News for hackers. Stuff that matters.
Re:voyage-code-3 by qeternity on Monday January 13, 2025 @09:11PM (#42692627)
Their 1024 dimension outperforms their 2048 dimension? What am I missing here.
Re:voyage-code-3 by fzliu on Monday January 13, 2025 @09:15PM (#42692659)
For float and int8, 1024 does indeed outperform 2048. However, for binary, 2048 outperforms 1024.

For our general-purpose embedding model (voyage-3-large), embedding vectors with 2048 dimensions outperform 1024 across the board: https://blog.voyageai.com/2025/01/07/voyage-3-large/

Re:voyage-code-3 by qeternity on Tuesday January 14, 2025 @11:24AM (#42699324)
Do you have any insights into this? Perhaps scaling degradation with Matryoshka learning?