News for hackers. Stuff that matters.
Re:Considering C99 for Curl by Karellen on Tuesday November 22, 2022 @09:08AM (#33705906)
The tipping point for me would be `snprintf()` and related functions. I've found it generally more useful/memorable/readable than strncpy()/strlcpy() and other updates to the dangerous and deprecated strcpy() just for copying strings safely, never mind it's other formatting abilities.

If Curl already has its own "decent and functional replacement" for `snprintf()` that's used extensively throughout the codebase, or if they just don't need that functionality (I haven't checked) then I guess that's not an issue. But that would be the big selling point as far as I'm concerned.

Re:Considering C99 for Curl by tialaramex on Tuesday November 22, 2022 @09:30AM (#33706161)
Note that strncpy() is not intended for safety. The purpose of strncpy() is to write to fixed size data structures such as part of the filesystem where you don't want to store NUL termination on strings.

Like 1980s Internet protocol features the rationale for weird things in C is more often "That's how Unix works" than "This is actually a clever safety feature".

Re:Considering C99 for Curl by dizzant on Tuesday November 22, 2022 @10:23AM (#33706747)
Is the original intent of strncpy() germane to the GPs comment? Explicitly stating the max length to copy is an effective tool for avoiding buffer overruns, regardless of whether the designers imagined that important use case.
Re:Considering C99 for Curl by tialaramex on Tuesday November 22, 2022 @10:56AM (#33707112)
But the thing it does (fill out a fixed sized buffer without caring about NUL-termination) is not at all what you'd want from a safety feature.

If you look at this function assuming it's a safety feature, that's a huge surprise, and indeed if you were skimming you might miss what it does because (in the context of "it's a safety feature") this is an insane choice. "Why would you do that?". Well, because it's not a safety feature.

The perf cost isn't what you'd expect from a safety feature either. Suppose we have a 1MB buffer, and we strncpy "THIS" into it using n = 1024. That's just four bytes right? Nope. strncpy() will write "THIS" and then 1020 zero bytes.

Re:Considering C99 for Curl by torstenvl on Tuesday November 22, 2022 @11:24AM (#33707490)
The issue is that strncpy() isn't a str___() function, despite its name. It's a 0x00-padded memcpy.
Re:Considering C99 for Curl by masklinn on Tuesday November 22, 2022 @11:36AM (#33707636)
But it's not an str* function, it's an strn* function. And most (though not all, that would be too easy) work on fixed-size (hence the n) nul-padded strings.
Re:Considering C99 for Curl by torstenvl on Tuesday November 22, 2022 @12:03PM (#33707990)
No. It isn't. It has absolutely nothing to do with strings. There is no such thing as a fixed-sized string in C. "A string is a contiguous sequence of characters terminated by and including the first null character." § 7.1.1.

Calling your bespoke byte-sequence data structure a "string" is inaccurate and viewing strncpy() as a string function is erroneous and will lead to buffer overflows.

Re:Considering C99 for Curl by masklinn on Tuesday November 22, 2022 @12:14PM (#33708147)
Have you considered giving reading comprehension a try?

Fixed-size strings are strings, just not nul-terminated strings. So are length-prefixed (“pascal”) strings.

Strncpy does not work with nul-terminated strings as I noted. If you’re working with fixed-size nul-padded strings however it’s the correct function to use.

Re:Considering C99 for Curl by torstenvl on Tuesday November 22, 2022 @12:16PM (#33708187)
> Have you considered giving reading comprehension a try?

Don't do this.

Re:Considering C99 for Curl by masklinn on Tuesday November 22, 2022 @01:46PM (#33709396)
Then don’t demand it by wilfully misunderstanding comments in order to “well actually” them.
Re:Considering C99 for Curl by cesarb on Tuesday November 22, 2022 @12:15PM (#33708163)
The output side of strncpy() might not be a str___() function, but AFAICS the input side of strncpy() is clearly a str___() function, since it stops reading (but not writing) at the first NUL byte.
Re:Considering C99 for Curl by masklinn on Tuesday November 22, 2022 @12:21PM (#33708255)
It’s incidental more than purposeful, though it does have its use as a conversion function: with a nul-padded field, why would you keep copying once you’ve hit the first nul? You can just set from an immediate without the need to read from memory.
Re:Considering C99 for Curl by masklinn on Tuesday November 22, 2022 @11:35AM (#33707622)
Except strncpy is broken for C strings, because it doesn't guarantee nul termination. So if you forget to force a termination on every use, you get buffer overruns.

Not only that, but (because of its actual purpose) it also fills the buffer with nuls, which is a complete waste of resources.

So yes, the original intent of strncpy() germane to the GPs comment, because it makes strncpy actively dangerous and complete shit when working with C strings.

Re:Considering C99 for Curl by kzrdude on Tuesday November 22, 2022 @10:46AM (#33706986)
Do we have a source for what the intended purpose is? I think you speak well to the effective purpose, but I'm not sure if it was that clear when it was introduced.
Re:Considering C99 for Curl by masklinn on Tuesday November 22, 2022 @11:41AM (#33707711)
> I think you speak well to the effective purpose, but I'm not sure if it was that clear when it was introduced.

It was completely clear, and can easily be inferred from its specified behaviour.

It's just completely useless nowadays, because its purpose is essentially obsolete, because the data type it works with is almost never used anymore.

strncpy works with fixed-size nul-padded fields as you'd find in e.g. mainframe-type software. That is why it:

- fills the destination buffer with NULs if the source is shorter

- does not nul-terminate if the source is the same size or longer than the destination

strncpy is essentially equivalent to zero-ing a buffer of size `n` then copying the first `n` bytes of src (up to the first nul) in the target

Re:Considering C99 for Curl by kzrdude on Tuesday November 22, 2022 @12:01PM (#33707965)
That's your understanding now. Fine, but not evidence of what someone else thought about this some decades ago.

(I don't use strncpy and don't defend its functionality, just want to know what was intended when it was introduced.)

Re:Considering C99 for Curl by kzrdude on Tuesday November 22, 2022 @12:07PM (#33708049)
I found an actual quote of a source which verifies the intention for fixed-length fields: https://softwareengineering.stackexchange.com/a/438090
Re:Considering C99 for Curl by spc476 on Tuesday November 22, 2022 @05:04PM (#33712187)
On early Unix systems, the directory structure was a simple 16-byte record; 14 bytes for the name, and 2 bytes for the inode. [1] strncpy() was used to simply record the file name into this structure.

[1] "UNIX Implementation" by Ken Thompson, _The Bell System Technical Journal_, July-August 1978, Vol 57, No 6, Part 2, pg 1942.

Re:Considering C99 for Curl by tjoff on Tuesday November 22, 2022 @12:04PM (#33708005)
Many protocols still relevant today make use of that structure, it still has widespread use.
Re:Considering C99 for Curl by masklinn on Tuesday November 22, 2022 @01:02PM (#33708742)
For the average protocol you're going to init the entire message then set into it, you don't need to fully zero fields.

This is mostly relevant to write into existing memory or memory-mapped records.

Re:Considering C99 for Curl by tjoff on Tuesday November 22, 2022 @01:08PM (#33708804)
Sometimes yes, but you must also not null-terminate the strings in those cases.

And on embedded devices you are often memory constrained so you might reuse an existing structure.

Re:Considering C99 for Curl by lemmiwinks on Tuesday November 22, 2022 @10:53AM (#33707068)
Re:Considering C99 for Curl by kzrdude on Tuesday November 22, 2022 @12:00PM (#33707936)
Doesn't say anything about the intention as understood when it was introduced
Re:Considering C99 for Curl by kazinator on Tuesday November 22, 2022 @11:17AM (#33707385)
> write to fixed size data structures such as part of the filesystem where you don't want to store NUL termination on strings

... AND where you want to pad the remaining space with zero bytes, so that you don't leak uninitialized memory onto the disk, or network.

The null byte padding behavior of strncpy makes it clear what the intended use was.

Also, the way C initializes character arrays from literals has strncpy-like behavior, because the entire aggregate is initialized, so the extra bytes are all zero:

   char a[4] = "a";      // like strncpy(a, "a", 4);
   char b[4] = "abcd";   // like strncpy(a, "abcd", 4);
the compiler could literally emit strncpy calls to do these initializations, so we might say that strncpy is a primitive that is directly relevant for run-time support for a C declaration feature.
Re:Considering C99 for Curl by synergy20 on Tuesday November 22, 2022 @11:32AM (#33707595)
then why not just use memmove() instead of strncpy() if no-NUL is the goal? not to mention memmove() is overlap-safe.
Re:Considering C99 for Curl by masklinn on Tuesday November 22, 2022 @11:39AM (#33707684)
> then why not just use memmove() instead of strncpy() if no-NUL is the goal?

no-nul is not the goal of strncpy, it's the effect of strncpy.

strncpy is designed to work on fixed-size, nul-padded strings. That's why it fills the destination buffer with nuls if the source is too short, and it doesn't guarantee nul-termination (if the source is exactly the size of the destination).

Re:Considering C99 for Curl by anthk on Tuesday November 22, 2022 @11:58AM (#33707918)
Get strlcpy and strlcat from OpenBSD.
Re:Considering C99 for Curl by pjmlp on Tuesday November 22, 2022 @01:58PM (#33709613)
Any version that still relies on separate parameters is unsafe, no matter what.

Some typo on the buffer limits and the same hazards as always.

Only fix is hardware memory tagging.

Re:Considering C99 for Curl by anthk on Tuesday November 22, 2022 @05:12PM (#33712297)
Multics?
Re:Considering C99 for Curl by skissane on Tuesday November 22, 2022 @10:44PM (#33714901)
I don’t think Multics did hardware memory tagging.

Systems which do/did include Burroughs Large Systems (now Unisys ClearPath MCP), IBM System/38 and AS/400 and IBM i (the RISC versions of which used PowerPC AS Tagged Memory Extensions), ARM MTE, SPARC ADI, and CHERI/ARM Morello.

Re:Considering C99 for Curl by pjmlp on Wednesday November 23, 2022 @03:12AM (#33716467)
Multics didn't need it, because PL/I does bounds checking by default, it has proper string and array data structures.
Re:Considering C99 for Curl by int_19h on Tuesday November 22, 2022 @11:07PM (#33715048)
C needs to bite the bullet and adopt slices as first-class types, so that they can be optimized on ABI level.
Re:Considering C99 for Curl by jstimpfle on Wednesday November 23, 2022 @01:11AM (#33715819)
How would you optimize slices the ABI level? Supporting them in function calling conventions should be easy. But figuring out storage representations I suppose would be a huge can of worms. There are too many ways of encoding slices depending on the use case. It only starts with the choice of a length field type (8, 16, 32, 64 bit. signed or unsigned)? There are also other representations thinkable, like sentinel values (NUL terminator) or more implicit storage of the size. Supporting them all in the compiler is not possible in practice.
Re:Considering C99 for Curl by pjmlp on Wednesday November 23, 2022 @03:12AM (#33716472)
Like every other systems programming language with a slice like feature, including those that predate C.
Re:Considering C99 for Curl by jstimpfle on Wednesday November 23, 2022 @04:34AM (#33717025)
Please show, don't tell. Your sibling explained that they meant only function call optimization. And I would say it's debatable that this is an "optimization" since slices would be a new concept that is distinct from structs. I agree though that the obvious naive choice would be to pass them in the same way that structs of { ptr, len } are passed in the ABI.
Re:Considering C99 for Curl by pjmlp on Wednesday November 23, 2022 @11:11AM (#33721007)
Infosec people have been showing the C folks for decades, showing alone isn't enough, when people refuse to change their habits.
Re:Considering C99 for Curl by int_19h on Wednesday November 23, 2022 @04:23AM (#33716934)
"Slice" is, by now, a fairly established term in PL design which implies a tuple of (start, end) or (start, length), so it specifically excludes prefixed length, null termination etc - because experience has shown that slices are the only sane choice.

What I meant by optimization is not treating them same as other structs, but e.g. guaranteeing pass-by-register like other primitive types, spelled out explicitly in the ABI. The choice of length field type would be size_t, obviously.

Re:Considering C99 for Curl by tssva on Tuesday November 22, 2022 @09:36AM (#33706230)
Curl uses its own implementations, curl_msnprintf and curl_mvsnprintf.
Re:Considering C99 for Curl by camel-cdr on Tuesday November 22, 2022 @10:41AM (#33706938)
I tend to stay away from the entire C style strings approach in general whenever I write C.
Re:Considering C99 for Curl by cassepipe on Tuesday November 22, 2022 @10:59AM (#33707143)
I always use antirez's SDS string Library. The fact that they are compatible C strings with the only price to pay being a call to sdsfree() instead of free I like a lot. Simple and yet super useful. Check it out.
Re:Considering C99 for Curl by anthk on Tuesday November 22, 2022 @11:59AM (#33707925)
Strlcpy and strlcat from OpenBSD works.
Re:Considering C99 for Curl by camel-cdr on Tuesday November 22, 2022 @12:04PM (#33708011)
IMO you should store the size of your strings. If you know the sizes already then you can just memcpy/memmove.
Re:Considering C99 for Curl by pjmlp on Tuesday November 22, 2022 @01:59PM (#33709624)
Only for those that never do mistakes with parameter passing.
Re:Considering C99 for Curl by kazinator on Tuesday November 22, 2022 @11:16AM (#33707373)
But snprintf can be used in C90. You just detect if it's available and use it, as if it were any other platform-specific function.
Re:Considering C99 for Curl by ben0x539 on Tuesday November 22, 2022 @07:53PM (#33713676)
Hm, why do people choose to do that instead of using the fallback case unconditionally? Expectation that platform-provided snprintf is faster?
Re:Considering C99 for Curl by kazinator on Tuesday November 22, 2022 @08:38PM (#33714006)
The platform-provided snprintf is smaller; it takes up zero bytes in your program.
Re:Considering C99 for Curl by masklinn on Tuesday November 22, 2022 @11:37AM (#33707647)
> But that would be the big selling point as far as I'm concerned.

I mean, it's a trivially replicated function, you can just copy it over from an existing codebase. So it's not exactly a major feature.

Re:Considering C99 for Curl by attractivechaos on Tuesday November 22, 2022 @11:40AM (#33707701)
The printf() family is much slower than direct string operations. snprintf() is not a good substitution when string copying is frequent.
Re:Considering C99 for Curl by dheera on Tuesday November 22, 2022 @11:57AM (#33707900)
> The tipping point for me would be `snprintf()` and related functions

Functions can always be implemented with additional header files.

They aren't actual extensions to the language.

Re:Considering C99 for Curl by unwind on Tuesday November 22, 2022 @03:04PM (#33710656)
If anyone reading this and feels uncertain on how this is done, DON'T do this:

    snprintf(dest, sizeof dest, source);   // BAD code do not repeat
that looks great at first sight, just another size-checked way of copying strings, but remember that the third argument to `snprintf()` [1] is of course a `printf()`-style formatting string. So if that `source` argument contains any percent symbols, there's gonna be a party in your computer and both Undefined and Behavior are going to show up. You don't want that.

Instead, if you want to use `snprintf()` for this, remember to do:

    snprintf(dest, sizeof dest, "%s", source);

[1]: https://linux.die.net/man/3/snprintf
Re:Considering C99 for Curl by Karellen on Tuesday November 22, 2022 @03:44PM (#33711248)
Thanks for mentioning this. I didn't include it in my comment because I originally thought it was too obvious, but considering it now, it probably was worth making explicit.
Re:Considering C99 for Curl by unwind on Tuesday November 22, 2022 @04:07PM (#33711521)
I blame Stack Overflow for conditioning me into seeing more of the possible ways things could break, when it comes to C code. :)
Re:Considering C99 for Curl by jbboehr on Tuesday November 22, 2022 @11:34PM (#33715216)
`-Wformat-security` [0] to the rescue!

[0]: https://fedoraproject.org/wiki/Format-Security-FAQ

Re:Considering C99 for Curl by hyc_symas on Tuesday November 22, 2022 @08:06PM (#33713760)
strlcpy() doesn't fix the problem of getting the length parameter wrong. The right solution is this:

    char *stecpy(char *d, const char *s, const char *e)
    {
      if (e) e--;
      while (d < e && *s)
        *d++ = *s++;
      if (d)
        *d = '\0';
      return d;
    }

    main() {
      char buf[64];
      char *ptr, *end = buf+sizeof(buf) ;
    
      ptr = stecpy(buf, "hello", end);
      ptr = stecpy(ptr, " world", end);
    }
As discussed here https://twitter.com/hyc_symas/status/1382298601641152513

The point is that the end of the buffer is invariant, there's no reason to screw around recalculating the length of remaining space after each copy into the buffer. This also fixes the nonsense of strcpy/strcat returning the same dst pointer that was passed in. By returning the pointer to where copying ended, you don't need a separate strcat function any more, nor do you have the Shlemiel The Painter problem with strcat.