News for hackers. Stuff that matters.
Re:Best aspects of C by asveikau on Tuesday November 17, 2020 @04:09PM (#25129879)
How about with joiners and combining characters? Eg. If you encode é as U+0065, U+0301 (\x65\xcc\x81), then search for 'e' and act on the result somehow, you fail to consider the whole glyph.
Re:Best aspects of C by loeg on Tuesday November 17, 2020 @04:43PM (#25130308)
Sure. You're talking about glyphs that are composed of multiple unicode codepoints; my earlier comment is true of single codepoints only. The comment I was responding to is also talking only about single codepoints (wcspbrk cannot represent delimiters longer than a single codepoint).

On joiners / combining characters: I'd encourage using composed normalization (NFC) rather than decomposed normalization (NFD).

Just curiosity: are there any glyphs that lack a single codepoint representation, where one of the joined codepoints is an ASCII character? (That only helps after normalization, of course.)

Re:Best aspects of C by jesuscyborg on Tuesday November 17, 2020 @05:15PM (#25130651)
Yes. ASCII uses \b as the combining character mark which is a convention that's always been widely supported by typesetting programs such as less and nroff. For example, A\b_ is A̲, and you can do the same thing with apostrophe and tilde for accent marks. There's also UNICODE emojis where two codepoints in sequence get joined together as a single glyph. Never underestimate the creative ways text can be used, or that standards just codify a long history of practices.
Re:Best aspects of C by loeg on Tuesday November 17, 2020 @07:37PM (#25131772)
Er, I was asking about unicode joining, not this roff \b thing. Sorry for the confusion. I'm aware that multiple-codepoint unicode glyphs exist; I'm asking if any of those involve a codepoint in the ASCII (1-127) range which cannot be normalized to a single codepoint (e.g., e + ' normalizes to a single codepoint é).
Re:Best aspects of C by jesuscyborg on Tuesday November 17, 2020 @08:08PM (#25132015)
Of course. Take for example mͫ (m+m) there's no way to represent that as a single codepoint. Combining marks can also be overlaid multiple times, e.g. m͚ͫ (m+m+∞) so the number of glyphs you can create is limitless. There's only a tiny number of the combinations that are possible which have a tinier normalized form. The new UNICODE combining marks work by almost exactly the same principles as the \b ASCII combining mark. That's why I mentioned it earlier.
Re:Best aspects of C by loeg on Tuesday November 17, 2020 @08:43PM (#25132227)
Thanks!
Re:Best aspects of C by kazinator on Tuesday November 17, 2020 @09:04PM (#25132345)
In what data format or programming language is 'e' a delimiter? One situation is floating-point constants, where 'e' is a delimiter indicating the exponent. However, if an é occurs in the middle of such a constant, whether as a single code point or a combined character, that is an error. The 'e' must be followed by an optional sign and one or more decimal digits.

The ISO C library string handling stuff is for systems programming, not for scanners and parsers for natural written language.