Re:Best aspects of C
by kazinator on Tuesday November 17, 2020 @08:58PM
(#25132316)
strstr can find a multi-byte character in a multi-byte sequence. There is no ISO C function to find the first occurrence of any one of a bag of multi-byte characters, though it's not difficult to write one.
> I have been told it doesn't make sense to break on whitespace for Chinese text.
It could make sense to break on whitespace in some programming language or data format that allows Chinese (and other) identifiers.
A command interpreter that allows Chinese arguments (such as file names) wants to break on spaces, as usual.
> if a delimiter appears as part of a multi byte sequence, you may see strange results.
UTF-8 was designed by a dyed-in-the-wool C-and-Unix engineer, who ensured that such a thing can't happen. No character in the 0x00-0x7F range can occur in a multi-byte character.
The byte which starts a UTF-8 character cannot occur anywhere other than at the start, which is why we can use strstr to look for it.
> I have been told it doesn't make sense to break on whitespace for Chinese text.
It could make sense to break on whitespace in some programming language or data format that allows Chinese (and other) identifiers.
A command interpreter that allows Chinese arguments (such as file names) wants to break on spaces, as usual.
> if a delimiter appears as part of a multi byte sequence, you may see strange results.
UTF-8 was designed by a dyed-in-the-wool C-and-Unix engineer, who ensured that such a thing can't happen. No character in the 0x00-0x7F range can occur in a multi-byte character.
The byte which starts a UTF-8 character cannot occur anywhere other than at the start, which is why we can use strstr to look for it.