Re:You can't just assume UTF-8
by josephg on Tuesday April 30, 2024 @09:06AM
(#40210601)
After reading the above comment I went looking for Unicode documentation talking about the different normalisation formats. That was a point I read that surprised me because I hadn’t ever thought of it. They said search should be insensitive to normalisation form - so generally you should normalise all text before running a search.
That’s a great tip - obvious in hindsight but one I’d never considered.
That’s a great tip - obvious in hindsight but one I’d never considered.