Re:Finnish breaks natural language processors
by yorwba on Friday March 01, 2019 @06:04PM
(#19286174)
If I do echo "忘れちゃった" | mecab, I get
忘れちゃ 動詞,*,母音動詞,タ系連用チャ形,忘れる,わすれちゃ,代表表記:忘れる/わすれる 付属動詞候補(基本) 反義:動詞:覚える/おぼえる
った 接尾辞,動詞性接尾辞,子音動詞ワ行,タ形,う,った,代表表記:う/う
EOS
So MeCab most definitely knows about that grammar. It's special-cased even! Admittedly, the way った is split off as a separate suffix is a bit ugly.
Re:Finnish breaks natural language processors
by Grue3 on Saturday March 02, 2019 @03:07AM
(#19288203)
That's the thing though, 忘れちゃ is a completely different, unrelated form that means 忘れて+は. So yes, this split is in fact incorrect and if it wasn't past tense, but dictionary form 忘れちゃう, the result would be even worse with う being split off as "rain"? or something.
[1] https://ichi.moe/cl/qr/?q=%E5%BF%98%E3%82%8C%E3%81%A1%E3%82%...