News for hackers. Stuff that matters.
Re:The dangers of streaming across versions of glibc: A caut... by cbsmith on Tuesday January 08, 2019 @04:15PM (#18859779)
I'm sorry but no.

It's a pretty common use case (so common that it is usually taught in an introduction to databases) that you might want an index that allows you to efficiently search by non-ASCII strings like say, a person's last name.

Re:The dangers of streaming across versions of glibc: A caut... by nine_k on Tuesday January 08, 2019 @05:17PM (#18860300)
I'd posit that this is not universally true. It may be true for English names.

It may be implementable for e.g. French names: a name like "François" may be stored as UTF-8 with "ç" always represented as a composite pair (or always a single character), and the application layer knows and uses this.

It must be a dubious idea for German names when you may need to see "Müller" and "Mueller" as the same name, but also keep e.g. only "Mueller" as the form referenced in legal papers.

Once you start considering anything non-ASCII, things become hairy. (Of course, if you only work for the US market and don't care about any international travelers as customers, you can continue using these introductory courses as a guidance.)

Re:The dangers of streaming across versions of glibc: A caut... by BeeOnRope on Tuesday January 08, 2019 @06:11PM (#18860717)
Of course it's hairy - that's kind of the point of this whole discussion!

People who use databases naturally end up wanting to store Unicode strings, and people naturally want to query them in locale-sensitive ways. Your Mueller example is a fine example of this. Databases have deep support for this, and you can also build your own compromise, so to speak, on top of the database with ICU or another library if it doesn't meet your needs.

Re:The dangers of streaming across versions of glibc: A caut... by nine_k on Tuesday January 08, 2019 @09:53PM (#18861882)
Storing is fine!

Indexing is full of caveats, though.

Re:The dangers of streaming across versions of glibc: A caut... by cbsmith on Wednesday January 09, 2019 @02:03PM (#18867675)
Indexing is always full of caveats. :-)