Re:Postgres as a Search Engine
by indulona on Sunday August 25, 2024 @02:28AM
(#41344918)
Eh, why? Have you ever heard of transliteration?
[ Parent
| on HN ]
Because search in different languages needs to be aware of that language's peculiarities.
Including but not limited to things that English doesn't have or only has in vestigial forms like grammatical cases, complex word morphology, declensions etc.
For example, it took Google years before they properly implemented search in Russian. That allowed Russian Yandex to win Russian market.
Back atcha: Have you ever heard of stemming?
That'll be different depending on the language, you can't simply transliterate words to another language, the mapping may change things and the rules might be different.
For example, in English "manage / managers", are practically the same, but transliterating to Spanish can give you "gestionan / gerentes", which diverge very early on.
Alternatively you could use "gestor", which has the same stem. "Gestionar" and "gerente" have different origins, so no wonder they look very different.
But yes, most (all?) European languages have vastly more complex morphology than English.