News for hackers. Stuff that matters.
Re:Mimic – abusing Unicode to create tragedy by lazyjones on Friday October 23, 2015 @08:09AM (#10438054)
> It's really difficult to do the right thing here. If Greek question marks share code point with semi-colon, it obstructs search and replace for question marks.

Context is the key here. Greek text doesn't use the semicolon for other purposes and searching/replacing such single characters in source code is a terrible idea anyway (think comments, string literals...). So what is the prohibitive failure scenario here?

Indistinguishable (for humans) characters with different code points were a stupid idea, it's fine to abuse it in order to point out that fact.

Re:Mimic – abusing Unicode to create tragedy by TazeTSchnitzel on Friday October 23, 2015 @12:00PM (#10439432)
They're not necessarily indistinguishable. Line-break rules, directionality and typographical conventions (height/width/alternate glyphs) may differ between apparent homographs. And using the same code point would make distinguishing between, say, Latin and Cyrillic searches difficult. How would Google tell if you mean CCCP or СССР?
Re:Mimic – abusing Unicode to create tragedy by wodenokoto on Friday October 23, 2015 @09:50AM (#10438493)
I don't know enough about Greek to deeply comment on it. some people do consider using the same code point for apostrophe and citations problematic (it's definitely annoying when doing word segmentation) we also have breaking and non-breaking spaces, as well as tabs. We luckily didn't follow typewriter conventions and collapsed several alphabetic characters!

A semicolon is not considered sentence final, whereas a question mark usually is. This makes it easier for software to auto capitalize. So at least that's possible a use case.

There's also the possibility that Greek requires the top dot to be square or circle, meaning it might in fact have subtle differences in print.

Re:Mimic – abusing Unicode to create tragedy by thaumasiotes on Friday October 23, 2015 @02:27PM (#10440384)
> some people do consider using the same code point for apostrophe and citations problematic (it's definitely annoying when doing word segmentation)

And those people are a menace. "Ball bearings" is a single word with a space in the middle. The only way you're going to get reliable word segmentation is with a natural language parser and a lexicon with an entry for "ball bearing". At that point, you're already recognizing "aren't" as a word; the punctuation isn't really relevant.

On the other hand, if you're not particularly upset about messing up space-including words, there's no real reason to be upset about apostrophe-including ones either.

Re:Mimic – abusing Unicode to create tragedy by lazyjones on Friday October 23, 2015 @10:46AM (#10438884)
> A semicolon is not considered sentence final, whereas a question mark usually is. This makes it easier for software to auto capitalize. So at least that's possible a use case.

Such software will need and have a language setting anyway (e.g. for hyphenation). It doesn't have to and cannot rely on code points alone, so the characters (or rather, different uses of the semicolon) needn't have different codepoints.

Re:Mimic – abusing Unicode to create tragedy by retbull on Friday October 23, 2015 @03:12PM (#10440685)
What if someone is quoting something in a different language for a translation or similar?
Re:Mimic – abusing Unicode to create tragedy by DarkUranium on Monday October 26, 2015 @04:27PM (#10454215)
To be fair, that would break hyphenation as well if both languages use a Latin alphabet (give or take a few carons and such).