News for hackers. Stuff that matters.
Re:The dangers of streaming across versions of glibc: A caut... by Someone on Tuesday January 08, 2019 @12:09PM (#18857192)
ICU isn’t the solution. If a bug in its collation routines is found, it likely will be fixed. You don’t want that if retrieving your data requires perfect stability of string comparisons.

The only real solution is to have your own collation routines, test the hell out of them for every release (you don’t want a compiler bug or the fixing of a compiler bug to introduce a subtle change in your collation code. The truly paranoid also worry about bug fixes in hardware), and to never stop supporting a collation, once you have ever released a product using it.

That’s what many databases do and why, for example, SQL Server has two binary collations. https://docs.microsoft.com/en-us/sql/relational-databases/co...: ”There are two types of binary collations in SQL Server; the older BIN collations and the newer BIN2 collations. In a BIN2 collation all characters are sorted according to their code points. In a BIN collation only the first character is sorted according to the code point, and remaining characters are sorted according to their byte values. (Because the Intel platform is a little endian architecture, Unicode code characters are always stored byte-swapped.)”)

Re:The dangers of streaming across versions of glibc: A caut... by j16sdiz on Tuesday January 08, 2019 @12:31PM (#18857423)
IBM use ICU for DB2 collation. It have versioned collation that do bug-to-bug backward compatibly.
Re:The dangers of streaming across versions of glibc: A caut... by mjevans on Tuesday January 08, 2019 @06:33PM (#18860855)
Or they could just drag Windows in to the world of sanity and store everything as UTF-8 (at last) instead of the 16 variant... Then endien-ness and a bunch of other bugs no longer matter.
Re:The dangers of streaming across versions of glibc: A caut... by jjnoakes on Tuesday January 08, 2019 @12:26PM (#18857362)
I imagine the database would use versioned collation, so a fixed bug would have to be opted-into by whomever owns the tables, no?