News for hackers. Stuff that matters.
Re:Checked C: extension to C that adds static and dynamic ch... by epx on Wednesday March 14, 2018 @08:38PM (#16589618)
One thing that C should have is an automatic string type (more like a byte[] than a String) that knows its own size. A huge amount of programming bugs arise from the lack of strings, and I can't see the reason why they could not be supported even in small systems. Delphi/Object Pascal had something like 3 or 5 function pointers that support the string type (create, free, resize, things like that).
Re:Checked C: extension to C that adds static and dynamic ch... by admax88q on Wednesday March 14, 2018 @08:55PM (#16589691)
The funny thing is that the memory allocator typically knows the size of the string.

You call free(foo) not free(foo, 12) or free(foo, strlen(foo));

But there's no standard way to ask for this size. Why doesn't strcpy just check with the allocator about how much memory is available and refuse to write past that.

Re:Checked C: extension to C that adds static and dynamic ch... by adrianN on Thursday March 15, 2018 @01:10AM (#16590539)
Because the programmer can do pointer arithmetic, treat non-char[] memory as char[] memory and probably other weird stuff that makes this more complicated than it sounds, I guess.
Re:Checked C: extension to C that adds static and dynamic ch... by m_mueller on Thursday March 15, 2018 @02:54AM (#16590893)
just make a type where doing pointer math and other such use cases are a compiler error?
Re:Checked C: extension to C that adds static and dynamic ch... by adrianN on Thursday March 15, 2018 @04:04AM (#16591110)
Just use a language where types automatically know their size? I thought the point was to make existing code safer by exploiting information that's already there (e.g. the allocator knowing the size of things on the heap).
Re:Checked C: extension to C that adds static and dynamic ch... by m_mueller on Thursday March 15, 2018 @04:13AM (#16591143)
The point is that C could be made much safer to use with a few "simple" (at least seemingly) tweaks. C has great tooling support on all platforms, including embedded, so it would be a compelling advancement if the language standard were to include such a thing. The important thing would be to coerce such a string type to character arrays so it remains compatible with all libraries without casting.
Re:Checked C: extension to C that adds static and dynamic ch... by rootbear on Wednesday March 14, 2018 @09:05PM (#16589739)
Not every string is allocated from the heap. There is no way to know if a string passed to a function, for example, came from the heap or not.
Re:Checked C: extension to C that adds static and dynamic ch... by admax88q on Wednesday March 14, 2018 @09:19PM (#16589813)
That's true, although there's no reason the stack allocator couldn't record this information somewhere as well.

And if the memory allocator doesn't know about the string then you could just revert to the existing behaviour so you're no worse off.

Re:Checked C: extension to C that adds static and dynamic ch... by adrianN on Thursday March 15, 2018 @04:08AM (#16591126)
Clang offers a memory sanitizer that does a lot of extra checking. People don't ship their software with the sanitizer activated because the performance hit is nontrivial.
Re:Checked C: extension to C that adds static and dynamic ch... by xtrapolate on Thursday March 15, 2018 @06:38AM (#16591653)
You're free to do as you please here. You don't have to use malloc()/free(), just implement your own LIBRARY_Alloc()/LIBRARY_Free()/LIBRARY_GetMemorySizeBytes() :)
Re:Checked C: extension to C that adds static and dynamic ch... by chrisseaton on Wednesday March 14, 2018 @09:37PM (#16589911)
There's more ways to allocate memory than the heap and stack.
Re:Checked C: extension to C that adds static and dynamic ch... by stochastic_monk on Wednesday March 14, 2018 @09:52PM (#16589970)
What are you referring to? sbrk? mmap? I typically use either the heap or the stack myself.
Re:Checked C: extension to C that adds static and dynamic ch... by MarceColl on Thursday March 15, 2018 @04:22AM (#16591164)
Something like memory pools or manual memory management. Yes the memory pool will come from the heap or stack but the heap/stack allocator will have no idea what is going on in there.
Re:Checked C: extension to C that adds static and dynamic ch... by timClicks on Wednesday March 14, 2018 @11:27PM (#16590225)
Static memory is at the bottom of the address space, below the heap. But that would really only be relevant in the case of string literals unless I'm mistaken.
Re:Checked C: extension to C that adds static and dynamic ch... by unwind on Thursday March 15, 2018 @04:10AM (#16591129)
You can have a static buffer:

    static char buf[128];
and use strcpy() at any point to copy into it, so I don't think statically allocated strings must be literals.
Re:Checked C: extension to C that adds static and dynamic ch... by stochastic_monk on Thursday March 15, 2018 @09:52AM (#16592812)
There is a distinction between const and non-const static memory, though, in that the former can be stored in Read-Only Memory. Still, static memory is generally used for many fewer purposes than the stack or the heap, and if I understand correctly, sbrk is still working with the heap, just more manually.
Re:Checked C: extension to C that adds static and dynamic ch... by timClicks on Thursday March 15, 2018 @08:17PM (#16597652)
That's right. Allocators use the sbrk and mmap to manage the heap. IIRC high performance allocators tend to use mmap exclusively.
Re:Checked C: extension to C that adds static and dynamic ch... by admax88q on Tuesday March 20, 2018 @12:29AM (#16626075)
Okay but you don't have to implement a 100% solution to make 90% of existing code much safer.

How much C code is out there that only uses malloc/free? All of those programs could be made safer if the default string methods were able to prevent writing past the end of the array.

Re:Checked C: extension to C that adds static and dynamic ch... by adrianmonk on Thursday March 15, 2018 @01:30AM (#16590606)
And all of them know how much memory they've allocated.
Re:Checked C: extension to C that adds static and dynamic ch... by MarceColl on Thursday March 15, 2018 @04:26AM (#16591173)
Yes, but unless you expose at the language level an interface for all types of allocators and enforce it then a whole lot of allocation techniques like memory pools couldn't be used with strcpy or would have to use char* type of strings again.

The point is not that allocators don't know how much memory has been allocated, but that C has no idea about the underlying allocator, so it cannot ask it about the length of the string as it was proposed.

Re:Checked C: extension to C that adds static and dynamic ch... by Gibbon1 on Thursday March 15, 2018 @01:47AM (#16590666)
However that doesn't mean that you shouldn't be able to either

Would be nice to be able to tell if something is allocated on the heap or stack in a portable way.

As in being able to ask if and object is thread local or not. The language and run time could support that but studiously does not.

Just like the language could allow you to directly monkey with the ABI but does not.

Re:Checked C: extension to C that adds static and dynamic ch... by wruza on Thursday March 15, 2018 @04:47AM (#16591231)
That something could be allocated by custom allocator like new or g_slice_alloc. You can monkey with the ABI as numerous FFI libraries do, but it is not portable by definition, unless you’re using, well, portable FFI library. Personally I can’t see why string, abi, etc libraries must be included. Include GLib++/GObject++ then and use its actually large ecosystem — what problem are you trying to solve by blaming low-level? That low-level is too low? Do you need high-level low-level that everyone will obey next morning? Check Java?
Re:Checked C: extension to C that adds static and dynamic ch... by Gibbon1 on Thursday March 15, 2018 @10:14PM (#16598081)
> portable FFI library

I looked into that, not available on many many platforms.

Re:Checked C: extension to C that adds static and dynamic ch... by _RPM on Wednesday March 14, 2018 @09:35PM (#16589901)
strcpy has no access to the "allocator". It doesn't know about malloc, realloc, or free.
Re:Checked C: extension to C that adds static and dynamic ch... by iainmerrick on Thursday March 15, 2018 @04:07AM (#16591123)
Yes -- and that has turned out to be a bad design decision. It's somewhat convenient to use methods like strcat() to build up strings, but it also leads to very insecure code.

I think the inclusion of strdup() shows that it was a mistake to make C strings completely allocator-agnostic.

Re:Checked C: extension to C that adds static and dynamic ch... by robotresearcher on Thursday March 15, 2018 @02:00PM (#16595099)
Strings don’t have to fill the char array they live in. Overallocating a buffer is a common technique to avoid frequent reallocation.
Re:Checked C: extension to C that adds static and dynamic ch... by munin on Wednesday March 14, 2018 @09:52PM (#16589968)
We have tried to do this by adding a "null terminated string" type. Start in section 2.4 of the specification (https://github.com/Microsoft/checkedc/releases/download/v0.7...).
Re:Checked C: extension to C that adds static and dynamic ch... by userbinator on Wednesday March 14, 2018 @09:18PM (#16589808)
I've used something similar --- a null-terminated string, with a length prefix at a negative offset. Resizeable ones have an "allocated size" field before that one. This makes for strings that are both easy to manipulate and compatible with existing functions that only read a regular C string.
Re:Checked C: extension to C that adds static and dynamic ch... by zzzcpan on Wednesday March 14, 2018 @09:27PM (#16589859)
Re:Checked C: extension to C that adds static and dynamic ch... by userbinator on Wednesday March 14, 2018 @09:44PM (#16589944)
Good to see I'm not the only one who thought of that idea, although my implementation has a different order of the fields (and IMHO better, since it allows for statically allocated strings with only a length field --- a sort of subclassing.)
Re:Checked C: extension to C that adds static and dynamic ch... by nikanj on Wednesday March 14, 2018 @11:31PM (#16590239)
So, BSTRs? They’re common in old-timey COM apis
Re:Checked C: extension to C that adds static and dynamic ch... by watersb on Thursday March 15, 2018 @04:02AM (#16591101)
BSTR size field is at positive offset, IIRC.
Re:Checked C: extension to C that adds static and dynamic ch... by nikanj on Friday March 16, 2018 @01:31PM (#16602317)
You do not RC :)
Re:Checked C: extension to C that adds static and dynamic ch... by timClicks on Wednesday March 14, 2018 @11:28PM (#16590228)
So in a sense implementing Pascal strings in C?
Re:Checked C: extension to C that adds static and dynamic ch... by flukus on Thursday March 15, 2018 @02:25AM (#16590804)
Yes. But also "abusing" C's permissiveness with pointers to get pascal strings that interop with standard functions. It get's passed around as a pointer to a char array but the length function would return the value at that pointer - sizeof(size_T). In most languages this wouldn't be possible.
Re:Checked C: extension to C that adds static and dynamic ch... by _sdegutis on Wednesday March 14, 2018 @10:51PM (#16590132)
The point is to keep C small and flexible. It supports whatever kind of string implementation needed by the present environment, platform and algorithm.

That said, several “high-level” string libraries for C exist that make it safer, more efficient, etc. Some of them are even pretty clever, allocating extra space right before the C pointer to a null terminated C string that it gives you, so it can do its own bookkeeping.

Re:Checked C: extension to C that adds static and dynamic ch... by peterevans on Wednesday March 14, 2018 @08:56PM (#16589696)
The main issue is that there's already a lot of standard library support for classical strings--things that are just a pointer to a chunk of bytes, and no more; rewriting those functions would be destructive. The C standards group could add to the standard library, I suppose, but generally they are a bit reticent to do so.

It is not the case that C has a benevolent dictator with a particular vision and drive for the language. It's more like a small group of people who don't want to mess things up. And so not much changes. This is neither good nor bad to me--it simply is; and anyway, these days, you have quite a few options that do offer automatic strings that can also compile into machine code (Go, Rust I suppose?).

Re:Checked C: extension to C that adds static and dynamic ch... by Gibbon1 on Thursday March 15, 2018 @02:00AM (#16590717)
> rewriting those functions would be destructive

Getting rid of standard string functions would be the best thing to happen to the c language in the last 25 years.

There are two types of c programs. Those that scrupulously avoid standard string functions and brittle programs shot through with security holes.

Re:Checked C: extension to C that adds static and dynamic ch... by kbenson on Thursday March 15, 2018 @03:06AM (#16590935)
At the point when you have to convert between string types and can no longer easily use much of libc without conversions, why not just use a different language that's already gone much farther in the name of safety, usability or other features?

In other words, if your pseudo-c candidate already suffers from most the interop problems that Rust, D, Nim and Go do, why not just use one of those and at least reap the other benefits they provide?

Re:Checked C: extension to C that adds static and dynamic ch... by watersb on Thursday March 15, 2018 @04:01AM (#16591097)
Anything to help legacy C code base is welcome. Hard part is knowing whether touching decades-old, tested code is worth the risk.
Re:Checked C: extension to C that adds static and dynamic ch... by peterevans on Thursday March 15, 2018 @03:07PM (#16595654)
I mean, I guess it's the best thing if you want to wreak havoc and chaos? Which, perhaps, you do!

Your suggested dichotomy is, of course, a little bit false. But I'm sure you knew that when you wrote it down.

It's entirely possible to write secure programs in C, even with standard functions. Writing your own code does not somehow confer a level of security-consciousness that you lacked when sticking to strings.h. (It does give you a wonderful opportunity to write your own security holes that no one has discovered yet!)

I mentioned this somewhere else, but we're in a pretty good place right now with languages; we finally have really solid alternatives to C that can compile to machine code, in both Go and Rust.

Re:Checked C: extension to C that adds static and dynamic ch... by Gibbon1 on Friday March 16, 2018 @05:30PM (#16604175)
> It's entirely possible to write secure programs in C, even with standard functions.

You realize that most standard string functions are outright banned by organizations that care about security. As in you're not allowed to use them not even if you pinky swear to be 'careful'

https://msdn.microsoft.com/en-us/library/bb288454.aspx

Re:Checked C: extension to C that adds static and dynamic ch... by isaachier on Thursday March 15, 2018 @12:07AM (#16590344)
I think Fortran uses a number followed by a byte array to represent strings. This is the big argument with C which used null terminators. In retrospect, maybe Fortran was right after all.

Edit: Actually seems I confused Fortran and Pascal. Also, Fortran uses an internal length variable.

Re:Checked C: extension to C that adds static and dynamic ch... by Gibbon1 on Thursday March 15, 2018 @01:55AM (#16590697)
I think the failure of C to support a built in array type is exactly like Steve Jobs refusing to give up one button mice.

The reasons aren't technical or based on merit but on stubborn ideology.

Re:Checked C: extension to C that adds static and dynamic ch... by rs86 on Wednesday March 14, 2018 @08:52PM (#16589679)
Arrays are actually pointers to the first element... They get translated to pointer arithmetic: adding an offset to the first element. You can easily create a struct with one field for the size and another with the pointer to the first character
Re:Checked C: extension to C that adds static and dynamic ch... by kqr on Thursday March 15, 2018 @01:44AM (#16590652)
Semantically, arrays are different from pointers. They do, however, degrade into pointers in many situations, which causes that misunderstanding.
Re:Checked C: extension to C that adds static and dynamic ch... by petecox on Thursday March 15, 2018 @02:07AM (#16590740)
A language with an array type, e.g. Java, distinguishes between a 'null' array reference and an array of zero length. which could substitute in languages that don't have an option type; SOME([]) vs NONE

But a plain *char in C makes no such distinction.

Re:Checked C: extension to C that adds static and dynamic ch... by charliesome on Thursday March 15, 2018 @03:32AM (#16591011)
There is a difference. An empty string in C is a non-null char* pointer pointing to a zero byte. Passing a null pointer to strlen will crash.
Re:Checked C: extension to C that adds static and dynamic ch... by esrauch on Thursday March 15, 2018 @03:46AM (#16591052)
A zero-length array can't ever be a valid cstring at all since they have to be null terminated. I don't think that is actually relevant to the claim you are replying to, which is about arrays and pointers and not strings.
Re:Checked C: extension to C that adds static and dynamic ch... by andrewmcwatters on Thursday March 15, 2018 @11:12AM (#16593497)
That type of thinking led to C++, a language whose contributors do not know how to stop adding features.