A search engine that favors text-heavy sites and punishes modern web design

Re:A search engine that favors text-heavy sites and punishes... by soheil on Tuesday September 21, 2021 @01:00PM (#28607092)

Are the n-grams always at most n=2 bigrams?

[ Parent | on HN ]

Re:A search engine that favors text-heavy sites and punishes... by marginalia_nu on Tuesday September 21, 2021 @03:36PM (#28608954)

No, I actually count the n-grams as distinct words (up to 4-grams). The main limiter is for that is space, so I only extract "canned" n-grams from some tags.

I would first search for the bigram hello_world, that's an O(1) array lookup; as then documents merely containing the words hello and world (usually not a good search result), that's the algorithm I'm describing in the parent comment.

[ Parent | on HN ]

Re:A search engine that favors text-heavy sites and punishes... by soheil on Tuesday September 21, 2021 @04:09PM (#28609325)

Makes sense. Every time you insert a new URL for a word you have to update the ranges for every other word since the URL file will be shifted?

[ Parent | on HN ]