News for hackers. Stuff that matters.
Re:Will keyword (BM25, TD-IDF) be replaced for search by Neu... by jamesblonde on Wednesday October 12, 2022 @04:09PM (#33182378)
Amr Awadallah, cofounder of Cloudera, is arguing that Keyword Search will be replaced with Neural Search within the next five years. He’s not alone; a number of companies have emerged in this space recently (e.g. Deepset Haystack, Hebbia, etc.). Amr launched his new startup Vectara today. They claim their Neural Search as a service is as easy to use as Algolia, as scalable as Elastic, but "neural-first" leading to much higher semantic relevance (they are free for 15k queries per month). My question: do you really see neural search replacing keyword search, or do you simply see it as an extension?
Re:Will keyword (BM25, TD-IDF) be replaced for search by Neu... by kacperlukawski on Friday October 14, 2022 @07:16AM (#33202009)
Both kinds of systems may coexist - the question that remains open to me is how to combine them both. Vector search, however, provides tools for non-textual search capabilities. Things like querying images, videos, etc, are non-solvable with BM25 and similar methods.