I guess that doesn't only apply to C++, but also C. :)
From the first chapter, the Go code starts using strings as a short-cut to represent tokens . In other languages this is trivial because strings are very easy to create, resize, change, etc. Using C, this became an issue though, as using strings became a roadblock where I started having to implement different solutions rather than focusing my attention on the contents of the book.
2 - no enforcement that a null terminator actually exists in the string
3 - C brags about performance and is probably the slowest language to compute string length
4 - manipulating strings requires very carefull handling of buffers, usually forcing everyone to use the heap as easier way out
If you are going to mention Assembly as possibility, check ClearPath where there is no Assembly, NEWP has full control over the hardware stack.
Obviously, since many other languages that do so are implemented in C.
Those are not practical? Billions of dollars have been wasted on issues stemming from this...
Or is the insinuation that we are hand-wavy about it, and you doubt the existence and scale of the problem? It's a well researched, well established problem, known for almost half a century.
https://en.wikipedia.org/wiki/C_standard_library#Buffer_over...
https://security.web.cern.ch/security/recommendations/en/cod...
http://www.informit.com/articles/article.aspx?p=430402&seqNu...
https://randomascii.wordpress.com/2013/04/03/stop-using-strn...
https://courses.cs.washington.edu/courses/cse341/04wi/lectur... (null termination)
https://www.geeksforgeeks.org/why-strcpy-and-strncpy-are-not...
https://www.owasp.org/index.php/Reviewing_Code_for_Buffer_Ov...:
And let's not even get into format string issues...
I have a feeling that of all axes of performace C cares the most about memory overhead. Then the obvious idea is to have it at exatly one byte per "simple" string, and you get to pick the class of programs that can't get away with that default string type:
• One-byte terminator: complicated text-handling application with a lot of (longer than a couple of pointers on average) string slices.
• One-byte length: anything that needs strings longer than 255 chars.
And then of these two solutions you pick the obviously more general one. What could possibly go wrong?
Character arrays with open length, bound checked.
Naturally it requires better compiler support than C authors were willing to implement.
Which is still the case with many things in Go, a language of close origin to C (though this time not about strings).
I guess security is not relevant as infrastructure work.
> I guess security is not relevant as infrastructure work.
...
And nowadays C code is considered legacy, with C#, Rust and constrained C++ as the road to the future.
Strange argument. Javascript is widespread, COBOL was widespread, Windows is widespread, X86 is widespread. Widespread doesn't mean good. UNIX was a disaster, and the whole family of UNIX-like OS spent decades just for mitigate its errors and faults.
And our data through some JS monstrosity.
So?
All the listed weaknesses also have benefits. For instance it is easy to get a substring without need to copy.
But yes, many bugs in C software originate from spring buffer overflows.
To make substring in some other languages, you need to store pointer to beginning of the substring and length of substring.
To make substring in C, you need to store pointer to beginning of the substring and put '\0' into original string.
Plus a pointer to the beginning, plus a reference counter as the user expects it to manage lifetime. In C this is the user's job. Where they know the life time is guaranteed they can optimize.
That substring won't remain valid without copying it.
Unless it's not modified? And unless, when modified, that shouldn't be its new value?
C Strings (nul-terminated) are the right approach for static storage of small static strings (like strings literals in the source code) since they have low overhead, and "substrings" aren't second-class citizens.
For dynamically allocated strings that won't be modified after creation, the right approach is using a large memory chunk that is shared between many such strings, plus two indices for offset / length (or just offset if it's text that can be terminated with a sentinel).
Having a short string of about 10 characters allocated as a dynamic object in its own allocation is wasteful. Slow to allocate and has about 2x to 3x overhead. This approach isn't good for applications that store a large amount of data.
> O(N) is not the same as O(1).
Don't call strlen() in situations where the strings are large and you need to know the length ahead of time, and running time is paramount. Instead, store the length.
I can tell you that in one of my programs, the difference between garbage collected strings and optimized ones (actually, strings converted into unique integer handles immediately) for one of my projects (SAT solver in Java), handling a few million variables, was something like a second until completion vs a couple of minutes before the garbage collector finally dies due to lack of oxygen, losing all data computed up to that point.
At least in C it's quite obvious that strings are not trivial if you want both an intuitive way to work with strings, and high performance. The C++ std::string type is neither intuitive to work with, nor does it allow to write high-performance code.
For string processing it's really better to use another language with different trade-offs.
It has string literals, so yes it does.
> The C++ std::string type is neither intuitive to work with
Many would disagree.
> nor does it allow to write high-performance code
True, but only due to backwards compatibility with C - std::string operations have to add a null terminator for no other good reason.
Many would also agree, that means nothing. Personally i dislike C++'s strings... and the rest of STL, which i view as one of the worst standard library APIs in wide use.
That's about the least reason why std::string is inefficient.
As for string processing in general, I do agree that other languages are better suited.
How does utf8 make that difficult to answer?
When was the last time you iterated over a string of unicode points and said you know what would be handy right now? If these code points were split up into arbitrary and unusable bytes of memory.
I don't think is a huge problem per se, though, you can just use a string library.
See the confusion here for an example: https://stackoverflow.com/questions/308695/how-do-i-concaten...
snprintf looks like it is the easiest way out.
If what you say if you want mutable strings, many languages have those and you don't need anything like "NULL" to have them (and you can use a bytearray of the string in Python, though Unicode complicates this).
>The NUL character is in my experience not so terrible, you typically have null pointers at the end of a linked list or whatever as well and nobody complains about that
That's not the same thing at all. The linked list is comprised of structs with next fields, that can be null or point to something. Your program can handle either just fine, as both are valid cases (a linked list expects to find the NULL guard at the end but also expects a non-NULL next pointer if the node is not the last one, so will handle both).
OTOH, if an incoming string doesn't have a NUL byte your program will crash/corrupt memory/worse. On top of that, you need to remember it to add it/make space/for most string manipulations. Strings are not expected NOT to end with NUL, and when they don't there's no way you can mitigate it, except to set arbitrary limits to how many characters you consider.
how are linked lists different? if the last node contains garbage for its next pointer, the outcome will be exactly the same. it's a bit more rare to encounter an "unterminated" linked list, but I've seen it happen plenty of times deserializing a linked list from disk or if the programmer just forgot to initialize the pointer. c strings basically are linked lists with an implicit next pointer.
@aap_, I asked a similar question some time ago and got some answers, you can check the thread here: https://news.ycombinator.com/item?id=19302581
The direct answer I got was:
> I'm guessing because an off-by-one or an extra skip might mean you miss the end of the string and go off into la-la land feeding whatever garbage happens to be in memory to your parser? That would mostly be a C issue (as it has no string abstraction at all).
Well, if someone is not familiar, why do they read a subthread on the matter?
Shouldn't they better start with a tutorial on C/C strings?
Even if people on this thread gave arguments, how would they (not familiar with C and C strings) would evaluate them? They could be totally bogus.
Length not known, so prone to overflows at anytime, atrocious standard library, ... (and let's not even go into the Unicode situation).
You will find my nick on those archives at the C++/Pascal trenches side.
And yes, apart from the myriads of usual C++ leaky abstraction bugs that "just" cause wrong results, this includes segfaults.
Mainframes were written in a mix of Assembly and safe system languages (IBM i, z and Unysis ClearPath are the survivors).
8 and 16 bit home micros OSes were largely written in Assembly, and all system languages were in equal footing as platform guests.
That's not what I remembered. Back in 1989, I bought the very popular Microsoft C 5.1 compiler (in shrinkwrapped box) from the Egghead retail store. Egghead was basically a smaller version of CompUSA.
Also, non-UNIX publications like PC Magazine and BYTE had ads for Borland Turbo C. Watcom C was also a competitor.
In the late 1980s and early 1990s, if one wanted to write COTS software for DOS on IBM PC compatibles, it was either assembly or C Language. Other languages like dBASE/Clipper/4GL were for LOB apps instead of COTS. In 1985, both Lotus 123 v3 and MS Excel were written using C.
It was more like the early 1980s before the personal computer revolution when the C Language was mostly tied to UNIX.
Back in the Iberian Penisula, Turbo Basic, Turbo Pascal, TASM, NASM reigned on the PC for COTS.
Amiga demoscene was all about AMOS and 68000 Assembly.
I code since 1986, touched my very first C compiler in 1992.
By then I already had used Timex 2068 Basic, 48+ Basic, CP/M on +3 A, Z80, Turbo Basic, Forth, Turbo Pascal 3 - 6, Dbase III Plus, 8086 with TASM.
Thankfully also in 1992, I got hold of Turbo C++ 1.0, and I was settled on my opinion about C.
But that doesn't mean you can't have fun with it, or write good computer programs.
It just means that user inputted text is always going to be a pain.
If you want support for unicode etc, there are libraries.
I'm not sure what you mean by compatibility issues in this context?
This tends to be important for strings you're going to show to the user (or input you gather from them), maybe most of the C in the world isn't aimed at interfaces like that...
For other encodings, well you need to decode explicitly (or just go for wide chars). Shouldn't come as a surprise, and not a big deal either.
I'm not aware of any. I've never encountered a language that is more frustrating than C to manipulate strings in.
> Are you saying other languages don't have security exploits?
No, I'm saying they have fewer. Far fewer.
> And in 35 years of writing in C, I have only had a very few segfaults while writing new code but never in production
Good for you. In 25 years of writing C and working with dozens of other programmers writing C in that time, I've lost count of how many segfaults and memory corruption errors I've had to debug.
I don't miss those days and never want to go back.
> unless your knowledge is only from reading reddit headlines.
It is not. See above.
I'm sorry, did you just say C doesn't have a comprehensive library?! I agree with your other points (but posit that they are not always negative; some of the 'flexibility' is useful in the domains where C dominates. Most of the shortfall can be covered by tooling, linting and standards.
It's been almost a decade since I last used C, but I think the complaints are overblown. Some tools have sharper edges than others, but there are there to accomplish a goal.
Data structures? Lists, Hashtables, Maps, Strings, Refcounted pointers, Weak pointers, Ephemerons, iterators for mentioned? C library is a joke even by 1970 standards.
>Most of the shortfall can be covered by tooling, linting and standards.
The amounts of CVEs prove you wrong. Besides, a typical C-coder considers himself smarter than type checker, otherwise he would be using another language (ML, Rust, Ada, C++).
Anyway, the author makes a few good points that are important but often overlooked, but I don't think I'd agree with him on the safety aspect in section 6. Making a safe C implementation doesn't really appear possible. If you allow casting integers to pointers, how do you implement these saftey checks he's talking about?