There are many strings that are not user facing, which you expect to be of a certain nature, e.g. ASCII based protocols, and therefore you know what to do with them.
So the multi byte situation and strcpy, or std::string, or any other "standard" string function isn't really relevant as it's some other libraries duty.
the task of truncation and otherwise formatting UI strings is the preserve of the rendering layer(s).
Note that the Unicode grapheme cluster, word, sentence, and line break algorithms are locale specific. Now consider how often programmers casually truncate strings, even in high-level languages, without accounting for the locale.
C is okay — not fan-worthy, but okay — for one specific kind of program: a mostly-portable implementation of a real language in which to write every other kind of program. that’s not nothing: implementing a backend for every CPU-OS pair one wants to support is a pain, and a different skillset from writing, say, a web browser or text editor.
I wonder how much C has held back the development of computing.
It’s the C hackers who insist on using it for inappropriate applications who create the problems.
Anything string-heavy and internationalized is a terrible place for C, even if you have spent 30 years dancing through the C minefield and are sure you’ll get it right this time.
Newsflash, a few decades ago the majority of software was the minimalist C that they claim sucks less. There are many reasons we moved on from that world, security firmly among them.
Lisp is older than C. It didn't just hold computing back, it actively walked backwards.
And this is precisely what I want, in some cases. I use C when I need a low-level byte wrangling code that tightly interfaces with the operating system, or when I can't afford to allocate memory at will (like all effortless string-handling languages do under the hood), or when I don't want any runtime and need my code to behave like a shellcode, basically, or...
I guess C is still used because people find it useful. No need to be a fan of it.
This was my mantra for the last 30 years. But now, even the tiniest thing has 1GB, linux and virtual memory. Except for very edge cases, this excuse is rapidly dying
Coreutils should continue to do their job, and not need to be recompiled against whatever encoding is in fashion.
Then again, I was surprised to find support for literal encodings in C23. So perhaps my knowledge was even more outdated than I anticipated.
Can you expand on this? Why does it matter to keep code points and grapheme clusters together in the case of truncation? If you’re already truncating the string, then you can just copy as many bytes as possible. Then, later when you interpret that string, you’ll hit a malformed codepoint and ignore it. I guess what you might be getting at is that if you have a codepoint sequence, then you shouldn’t copy the bytes if they can’t all fit in the truncated string?
I feel like this is an edge case and not “the reason almost all C software has problems with non-English strings”. 99% of the time, copying up until the null byte is fine, whether or not the string is UTF-8 or ASCII. The reason most C software doesn’t work with non-English strings is because the developer never added support. The bytes are still there, they just need to be interpreted correctly in the UI portions of the code.
Truncating mid-grapheme-cluster can change semantics in unexpected ways. For example, the "family" emoji(s) can be encoded as a set of conjoined codepoints for each family member: https://stackoverflow.com/questions/49958287/printing-family.... Depending on where you truncate, you might just get the father, which probably won't cause you any serious issues but it might cause confusion depending on context.
(IMHO, if you're truncating a way where this would matter, you should probably be truncating in screen-space, i.e., way higher up the stack)
As far as this being the reason most C code doesn’t handle UTF-8 though, I’m still skeptical.
I mean if your viewpoint is to handle heavy UIs then yes maybe C is not your go-to language. However, there are other solutions to the character space problem besides UTF-8. Rather than complaining that your screwdriver can't bang nails, perhaps try using a screw?
No, you won't be able to support a language like chinese within a 256 bit charset, but perhaps it's not important to in all situations. The strength of ascii is that its small, simple. If that's too anglo-centric for you, maybe there's a better symbolic tiny charset that is more applicable. I'd support something like https://lojban.io/ becoming more prevalent, but there could be advantages to having a simple(r) symbolic transmission language not burdened with anglo-centric concepts.
UTF-8 is fine for a display encoding. However, not every string encoding need be a display encoding, which the parent post seems to not be considering.
You could also have multiple display encodings, if it makes sense to (a tool only intended for use in a certain part of the world for instance), however that is not what I mean.
So many C developer seem to think they're "smart enough" to handle strings correctly using only the standard library, not needing "training wheels" like the plebs - but then where do the countless string related memory safety errors and all the broken Unicode handling come from that plague every C program, no matter how clever the programmers, no matter how tough the review process, no matter how much static analysis is used?
The issue is that many C developers simply can't seem to admit that C string handling, as it was invented many decades ago, was simply fundamentally flawed. So easy to misuse with catastrophic results, even for plain 1-byte-strings. It has nothing to do with how "smart" you are or how careful you are.
I think it's more a case that programming is English skewed and localization concerns are rare for the programmers these day who still need to work in the realm of C (so, mostly for older legacy software or the embedded realm).
The result is that the same mistakes with string handling get repeated again and again, often with catastrophic results.
I mostly ask who's defending it for holistic purposes. Right or wrong, It would be nice to hear other perspectives on the matter if possible.
Unlikely to occur? Do you think that will put the naysayers at ease?
> , and from your comment I suspect you don’t actually know what that problem is (because you seem to be pointing at a different issue which is actually irrelevant). Maybe you or someone else can describe what the real concern is?
Amazing that people have the patience for this.
Especially because concatenation is a very common operation so those sliced codepoints will be everywhere, including in the middle of text.
a: there was a list somewhere over which characters hackernews allows?
Yes, many languages allow strings (UTF-8 or otherwise) to contain null bytes, and C's str*() functions obviously do not, but null-termination vs not is an orthogonal issue to ASCII vs UTF-8.
i.e. Yes it's (depending on context) an issue that C str*() cannot handle strings with embedded null bytes, but that's not a UTF-8-specific issue.
Also, by this rationale, NO string is ever safe in C, because pretty much every encoding technically supports codepoint 0 (even though you take your life into your hands should you ever use it). This is not a useful discussion.
This is not meant as a personal attack; I just want to point out how it looks on a casual reading :)
Which is?
It's also pretty telling that every article that tries to explain how to safely copy or concat strings in C, like this one, only ever works with ASCII, no attempt whatsoever to handle UTF-8 and keep code points together, let alone grapheme clusters. No wonder almost all C software has problems with non-English strings...