Re:Jina AI Launches First Open-Source 8K Text Embedding, Riv...
by rolisz on Thursday October 26, 2023 @01:35AM
(#38021995)
That's not quite tfidf though. I agree you can get better results than that with Ada embeddings, but I would argue you can get even better results with embeddings from smaller chunks.
[ Parent
| on HN ]
Re:Jina AI Launches First Open-Source 8K Text Embedding, Riv...
by simonw on Thursday October 26, 2023 @01:50AM
(#38022093)
I guess technically it's bm25, since it's using the rank mechanism in SQLite FTS5: https://www.sqlite.org/fts5.html#sorting_by_auxiliary_functi...