"Better Relevance"
and I am simply saying, arguably, you can't make that claim without evidence.
(That said, I do find much to like about the article and the WAND / T-WAND explanation. I'm only pointing out you can't claim 'better relevance' without a benchmark to go with it.)
A good top-K algorithm should rank a document containing more user query terms higher than a document containing less number of user query terms. This makes perfect sense. Right?" This should have been changed in: "For the sake of this blog post I am interested in a simplified relevance concept: more query terms matched -> more relevance" In this way the blog would have not caused the outburst of (correct) indignation of the Information Retrieval community, that has been working for years on the topic, and knows very well how much complex is relevance.
Why should I care about what they think? In the end of the day, they are serving the users. They are either going to be left behind, or they have to change.
For example, the newer search engines, such us, https://github.com/meilisearch/MeiliSearch, also do what T-Wand does. Users are going to vote by their feet. The "information retrieval community" be damned. It's as simple as that. Got it?
And change they will. What "information retrieval community" are you talking about? Just a bunch of fresh graduates who just got their PH.D, and some college students. That's about it. I don't see any IR professors coming out defending them.
That's just wrong. Period. This is the reason I mention his job, because his job requires him to listen to the users, instead of taking something he learned in school and use that to talk condescendingly to his users. That's not gonna fly.
Given this context, I think whatever I said it is entirely appropriate.
I'm just trying to understand what the author meant and their intention for anyone that might revisit this thread later. I think what you're saying sounds plausible as an interpretation.
I was simply motivate my work, pointing out it's a problem that the current generation of search engine does not address.
What is "relevance", it is of course a context sensitive question.
You talk as if BM25 is the gold standard when it is not.
The research on this is all over the place. I just read an article that says that BM25 is way worse than alternative language models. You don't have to look far, for example this one that talks about Wand:
https://dl.acm.org/doi/10.1145/2537734.2537744
You know why I quit academia? It is useless arguments and virtual signalings like these.
I'd rather go out and build a damn thing that people like to use.
I merely pointed out that there's a problem, and I have a solution. I did not claim that my problem and my solution solve all problems. Isn't this obvious?
In my problem, my solution beats Lucene. It's as simple as that.
So if Lucene wants to be this infinitely configurable search library, it would be advisable to offer my solution as an option, or offer an better one that does something similar.
So far I have not seen any takers, only excuses.