Re:Wink-lemmatizer
by afsina on Sunday June 24, 2018 @07:59AM
(#17386200)
Good luck with Turkish. A very large lookup table may work for %90 cases but it will still fail. A basic `true` lemmatizer requires either a complex graph with rules, or an FST generated from it. input is searched through the graph and morphological disambiguator must be applied to result to pick the correct lemma.
[ Parent
| on HN ]
Or Welsh. You want a canonical form for "wnaethpwyd"? Try "gwneud"! Some regexes and an exceptions list isn't going to cut it.
The more a language needs a lemmatizer for NLP the harder it is to write it.
Re:Wink-lemmatizer
by tomjakubowski on Sunday June 24, 2018 @11:51AM
(#17386971)
Could someone explain what's hard in particular about canonical forms in Welsh?