News for hackers. Stuff that matters.
Re:The dangers of streaming across versions of glibc: A caut... by cbsmith on Tuesday January 08, 2019 @04:13PM (#18859756)
It's not that simple. SQL in particular actually requires lexicographical sorting, so you need to do it somehow. The problem is relying on lexicographical sorting remaining fixed when it isn't. Per the unicode standard, lexicographical sorting rules are subject to change. So now you need a mechanism to detect/manage those changes.
Re:The dangers of streaming across versions of glibc: A caut... by rurban on Tuesday January 08, 2019 @06:15PM (#18860749)
That's done via

    unicode/uchar.h:
    #define U_UNICODE_VERSION "9.0"
in icu. pcre2 has PCRE2_CONFIG_UNICODE_VERSION 10 fribidi has FRIBIDI_UNICODE_VERSION "6.2.0"

libc's need something similar. In safelibc I do have nothing so far, but I'm at 11.0

Re:The dangers of streaming across versions of glibc: A caut... by cbsmith on Wednesday January 09, 2019 @02:18PM (#18867845)
Right... so properly detecting when U_UNICODE_VERSION in the replication target differs from U_UNICODE_VERSION in the source, and adjusting for it is what's needed. PostgreSQL wasn't doing this in 2014.
Re:The dangers of streaming across versions of glibc: A caut... by rurban on Thursday January 10, 2019 @06:12AM (#18873038)
Exactly. I'm just not happy with the types of these defines. Only major updates need to be tracked, so the PCRE format seems to be the best. But all others are using strings, not an integer.
Re:The dangers of streaming across versions of glibc: A caut... by antirez on Tuesday January 08, 2019 @04:21PM (#18859821)
Yes I understand that, that's why I said that it's not applicable to higher level systems. Btw to put my comment in context, also Redis has a similar feature to SQL DBs where it shows results in lexicographical sorting directly to the user, but it that case it just provides memcmp() byte-to-byte sorting. So what you do in order to index "Aèèé" is to apply a transformation to that string and store a transformed prefix and later the string itself, like "aeee:Aèéé" (ref: https://redis.io/commands/zrangebylex)
Re:The dangers of streaming across versions of glibc: A caut... by senderista on Tuesday January 08, 2019 @08:10PM (#18861421)
You’re talking about sort keys, no? That allows you to deal with collation in a higher layer (which generates the sort keys per desired collation), but delegate the actual sorting to a dumb lower layer (i.e., memcmp).

(Just because I can’t resist, I’ll mention that sort keys are a perfect application for Hu-Tucker codes: https://cs.stackexchange.com/a/49586).