News for hackers. Stuff that matters.

A comment on Unicode sorting is hard and why browsers added special emoji matching to regexp. Parent comment

Re:Unicode sorting is hard and why browsers added special em... by lifthrasiir on Monday June 28, 2021 @12:32AM (#27658154)
> [...] CJK, ideograms, emojis, and many other non-alphabetical languages do not have order.

Uh, no? Most if not all scripts do have letter order, CJK(V) "ideographs" included. The actual problem is twofold: the letter order might not be unique, and the word order might be completely different from the letter order. CJK(V) "ideographs" exhibit both problems---their word order is phonetic, while letter order is based on shapes and different dictionaries use different rules---and it is very common in Chinese and Japanese that the string meant to be sortable has pronunciation attached (e.g. Wikipedia).

Also, believe it or not, even emojis do have order too [1], albeit arbitrary one.

[1] https://unicode.org/emoji/charts/emoji-ordering.html

[ Parent | on HN ]
Re:Unicode sorting is hard and why browsers added special em... by naniwaduni on Monday June 28, 2021 @01:46AM (#27658509)
Even English a few edge cases where collation order doesn't match letter order: titles are typically ordered ignoring initial A/The, names are often sorted by surname (even when written with given name first); famously, M'/Mc are are traditionally sorted with Mac, along with a host of special cases that get gleefully thrown out in this era of doing very dumb very fast.
[ Parent | on HN ]