> Null-terminated strings are actually a very good choice of string representation that is to be directly manipulated by programs, instead of just through opaque handles via library functions.
You seem to be having it both ways.
As I see it, "NULL" is a valid ascii character, but not a valid element of a C string - except as a terminator external to the string of characters being represented. It's fundamentally broken as an abstraction for managing variable length strings of ASCII. It is incapable of representing all strings of ASCII characters.
There are 32 control characters, and DEL. The purpose of all those characters is to carry some kind of meaning. The designers of the null-terminated string decided that the meaning of NUL will be reserved for that termination. That still leaves 31 control characters and DEL which can be stored in a string, which programmers can use for other meanings. That turns out to be plenty.
The "unreasonable effectiveness" and ubiquity of null terminated strings shows that the world is perfectly capable of getting along without NUL in the middle of a string. Basically, the intuition that we can steal NUL for string termination was right (and to some extent self-fulfilling, due to the subsequent proliferation of the convention).
The idea that all possible code points of a character set are valid characters that must be stored in a string is outdated in the age of Unicode, which contains codes which are not character code points.
Null terminated strings are now the representation of a path name every major OS that has any non-negligible installed base. The choices leading to this situation weren't made by complete idiots.
And, by the way, NUL is the ASCII character; NULL is the pointer constant in C.
http://worldpowersystems.com/ARCHIVE/codes/X3.4-1963/page5.J...
This got shortened to NUL in 1965.
The flaw isn't the null termination but the lack of encapsulation.
No string representation is error-proof, if the application programmer open-codes the manipulations of it, taking on the responsibility for maintaining all necessary invariants of the representation, including memory management. If applications take on the responsibility of constructing and maintaining length + data strings, there will be bugs.
Null-terminated strings are actually a very good choice of string representation that is to be directly manipulated by programs, instead of just through opaque handles via library functions.
Null-terminated strings allow for elegant tail recursion like:
Compare with: This leverages the fact that if s is a non-empty string, then s + 1 is also a string, which is its suffix. This is super cool and has no equivalent in the length + data representation.Compilers can merge string literals that have a common suffix; if "pet" and "carpet" occur in the same program, they can share storage.
Null terminated strings of byte-wide characters can be transmitted with no byte order issues. Not so when length fields are involved.
Being able to split a null-terminated string in-place according to some delimiter characters, by overwriting them with null bytes, is also handy, and greatly simplifies some programs which open-code this logic rather than relying on string libraries.
Null termination is nice for lexical scanning. Why? Because every string has an accessible byte that is one position past its last character, and has a unique value which can treated as a member of the input alphabet, and thus integrated into the pattern matching logic. Code which scans a null-terminated string for a lexical pattern doesn't have to do anything ugly and extraneous like checking an index against a maximum length; it just contains cases that handle zero as a part of the syntax.
Stream scanning code that looks for and handles EOF return from getc can easily be turned into string scanning code if it just handles zero instead of EOF.