News for hackers. Stuff that matters.
Re:BCHS: OpenBSD, C, httpd and SQLite web stack by rnkn on Wednesday January 19, 2022 @12:29AM (#29990324)
Is there a good string-manipulation C library?
Re:BCHS: OpenBSD, C, httpd and SQLite web stack by avar on Wednesday January 19, 2022 @05:05AM (#29991780)
The strbuf library that's part of git.git is a pleasure to work with. It's C-string compatible (just a char /size_t pair), guarantees that the "char " member is always NULL-delimited, but can also be used for binary data. It also guarantees that the "buf" member is never NULL: https://github.com/git/git/blob/v2.34.0/strbuf.h#L6-L70
Re:BCHS: OpenBSD, C, httpd and SQLite web stack by MrBuddyCasino on Wednesday January 19, 2022 @06:49AM (#29992461)
This looks really well thought out - bookmarked.
Re:BCHS: OpenBSD, C, httpd and SQLite web stack by pjmlp on Wednesday January 19, 2022 @03:39AM (#29991270)
Yes, SDS from Redis project.

https://github.com/antirez/sds

However the moment you call into other C libraries, they naturally only expect a char *.

Re:BCHS: OpenBSD, C, httpd and SQLite web stack by lelanthran on Wednesday January 19, 2022 @04:08AM (#29991446)
I got tired of running into this problem and decided to simply eat the cost of using `char *` in my string library.
Re:BCHS: OpenBSD, C, httpd and SQLite web stack by pjmlp on Wednesday January 19, 2022 @04:29AM (#29991584)
And that is why most such efforts eventually die.

WG14 could naturally work into something like SDS for strings and arrays, but of course that is out of their goals to ever do that.

Re:BCHS: OpenBSD, C, httpd and SQLite web stack by lelanthran on Wednesday January 19, 2022 @05:14AM (#29991836)
> WG14 could naturally work into something like SDS for strings and arrays, but of course that is out of their goals to ever do that.

Maybe it is, but even if it were, sds strings are a poor choice. I used them extensively in a private project.

1. Typedef'ing `sds` to a pointer type. This leaves no indication to the reader of code that any `sds` typed variable needs an `sdsfree`. IOW, for every other standard type it is clear when the data object needs a `free`, `fclose`, etc. This is a big deal, it's difficult to change the typedef for sds due to the way it returns pointers.

2. Not compatible with current string functions, strike 1: storing binary data in the strings, like the nul character makes it silently lose data when used with current string functions that accept `const char *`. This is a very big deal!

3. Not compatible with current string functions, strike 2: an sds string is only compatible with current string functions that take a `const char *`. This isn't such a big deal (for example, it provides a replacement for `strtok` as the standard `sds` type won't work for `strtok`) but it's unnecessarily incompatible.

4. With the current way it's exposed to a caller, you cannot use `const sds` variables anywhere, which removes a lot of compiler-checking. Trying to use `const` on any sds variable is pointless as you get none of the error-checking.

While sds solves many problems with raw C strings, those problems can be solved by adding standard library functions that work with existing C strings. In addition, it adds a few more problems of its own.

Re:BCHS: OpenBSD, C, httpd and SQLite web stack by jstimpfle on Thursday January 20, 2022 @04:57AM (#30006870)
"C strings" really aren't anything worth talking about. People take them way too seriously and then complain that they are "unsafe" or "hard to use". Look, C gives you memory to work with and the rest is up to you. Almost the only thing you want from C with regards to strings is string literals.

It should be obvious that most "string" APIs from libc like strcat, strcpy, but especially strtok are ridiculously bad and are only in the libc because of history. Don't use them.

Even strlen() is rarely a good idea to use, and you can (should?) replace strlen("abc") by sizeof "abc" - 1.

Re:BCHS: OpenBSD, C, httpd and SQLite web stack by pjmlp on Wednesday January 19, 2022 @05:20AM (#29991880)
My point regarding WG14 wasn't to add SDS as they are, rather vocabulary types for strings and arrays in the same spirit as SDS.

When they exist as vocabulary types, the ecosystem can rely on their existence and slowly adopt their use, similarly to threads support introduction in C11, for example.

Re:BCHS: OpenBSD, C, httpd and SQLite web stack by lelanthran on Wednesday January 19, 2022 @05:27AM (#29991924)
> My point regarding WG14 wasn't to add SDS as they are, rather vocabulary types for strings and arrays in the same spirit as SDS.

Well, yes, I'd love to see some proper string support too, so at least we're in agreement about that :-)

But, overhauling C with additional (memory-safe) array types and string types that are nonetheless still compatible with legacy uses is probably a non-starter anyway. The only way forward would be to add a new type that isn't compatible, which is unpalatable to a lot of people (myself included).

Adding memory-safe functions and/or semantics is easier, but will probably not cover 100% of the memory-safety desired.

> When they exist as vocabulary types, the ecosystem can rely on their existence and slowly adopt their use, similarly to threads support introduction in C11, for example.

Threads, I feel, are a poor example for two reasons: 1) Hardly any code uses the `thread_t` type for a variety of reasons, and 2) There was no need for a `thread_t` type to be backward compatible with anything.

Re:BCHS: OpenBSD, C, httpd and SQLite web stack by pjmlp on Wednesday January 19, 2022 @07:53AM (#29992957)
For full memory safety with C the only option are the C Machines, meaning hardware memory tagging.

Already in use for a decade in Solaris SPARC, and eventually mainstream across all variations of ARM CPUs.

Unfortunely Intel botched their MPX implementation and now it is gone.

Re:BCHS: OpenBSD, C, httpd and SQLite web stack by jstimpfle on Thursday January 20, 2022 @05:15AM (#30007001)
Apart from plain old fixed buffers, which is what is supported by C just fine and which covers 99% of string processing needs in the areas that C as a language is suited for anyway, ... there are 14 known ways of doing "strings" depending on circumstance, so I don't think it would be a good idea to introduce one mandatory version of them into the C standard. There is already C++ which has std::string, and there are a lot of GC'ed and scripting languages that are more suited for quick and dirty string processing.
Re:BCHS: OpenBSD, C, httpd and SQLite web stack by int_19h on Thursday January 20, 2022 @06:29PM (#30016939)
The fact that C++ was able to eventually standardize on a single string type (despite the same mess of many dozens of incompatible implementations) shows that it is possible and desirable. It's not like raw buffers will go anywhere if you add a higher-level type. Nor does it have to be perfect - only "good enough" for use across the API surfaces of various libraries.
Re:BCHS: OpenBSD, C, httpd and SQLite web stack by jstimpfle on Friday January 21, 2022 @05:21AM (#30021593)
Just because it's possible to standardize on a string type in C it doesn't mean it's desirable. Also consider that it's not possible to copy C++'s string type because its ergonomics build heavily on RAII.

'const char *' arguments work just fine as parameters in libraries, and I don't see much of a use case (and insteaad more hazards) for a library that "resizes" a string argument destructively (like std::string does). The typical way to go about this is for the library to make a copy of the input string. On API boundaries, for memory that is needed longer than the function call lifetime, it is almost always an excellent idea to simply copy it. For data that doesn't make sense to copy (be it because of size or because only one side really needs it), the data should instead simply be created on the right side of the fence from the beginning.

I don't see myself needing a standardized string type because I'm not passing around string "objects", or concatening them, like it would be done in quick and dirty scripts. I honestly can't recall where that kind of thing would have been a good idea for my work in the last couple of years, and I'm much in favour of not growing the standard out of proportion. As said, if you desire C++ kind of ergonomics and want to solve more scripting-like tasks, there is already C++ and a ton of other languages.

What I can recall is skimming through a lot of C projects over the years that tries to do object-oriented and scripting type programming in C (often it wanted to be C++ or Java or even Python but it had to be C for some external reason), and that code is always, invariably, an unmaintainable mess where it's impossible to have a level of confidence that there are no memory errors and leaks. C is simply not suited for that style of programming.

Re:BCHS: OpenBSD, C, httpd and SQLite web stack by int_19h on Thursday January 20, 2022 @06:26PM (#30016899)
That's not really a problem if the only thing they need is direct access to a read-only view of the buffer (i.e. const char*) - then it's no different than C++ and std::string.
Re:BCHS: OpenBSD, C, httpd and SQLite web stack by lelanthran on Wednesday January 19, 2022 @03:57AM (#29991387)
> Is there a good string-manipulation C library?

You will have to define "good". My string library[1][2] is "good" for me because:

1. It's compatible with all the usual string functions (doesn't define a new type `string_t` or similar, uses existing `char *`).

2. It does what I want: a) Works on multiple strings so repeated operations are easy, and b) Allocates as necessary so that the caller only has to free, and not calculate how much memory is needed beforehand.

The combination of the above means that many common string operations that I want to do in my programs are both easy to do and easy to visually inspect for correctness in the caller.

Others will say that this is not good, because it still uses and exposes `char *`.

[1] https://github.com/lelanthran/libds/blob/master/src/ds_str.h

[2] Currently the only bug I know of is the quadratic runtime in many of the functions. I intend to fix this at some point.

Re:BCHS: OpenBSD, C, httpd and SQLite web stack by lanstin on Wednesday January 19, 2022 @02:26AM (#29990897)
No? Asking for code nav and you get three answers. Asking for this and you get crickets. In the 90s I worked at a place where we embedded TCL into all the apps, and rolled our own templating systems. I had to do a little string stuff in C after few years of go, and it sucked. Ugg. buf[len] = ‘\0’;

Using go, I thought I was getting back to low level stuff but this C experience made me appreciate strings in Go. Web servers in C are crazy bad idea, especially if they are spitting out html. Lisp would be better. Node would be better. Go would be better.

Re:BCHS: OpenBSD, C, httpd and SQLite web stack by foxfluff on Wednesday January 19, 2022 @05:37AM (#29991994)
> buf[len] = ‘\0’;

So why didn't you use one of the bazillion library functions or third party libraries that terminate strings for you?

I feel like most of the criticism is coming from people who punish themselves by rejecting library functions and then telling that strings are hard. Doh.

Re:BCHS: OpenBSD, C, httpd and SQLite web stack by lanstin on Wednesday January 19, 2022 @10:32AM (#29994887)
I like to see the terminating nul in there, just in case my math was off earlier. I had strdup and so on. I was just way more work than go. And I was writing a LD_PRELOAD that I didn’t want to drag in any extra dependencies other than libc. Trust me, it is faster and safer and more fun in Go than C.
Re:BCHS: OpenBSD, C, httpd and SQLite web stack by foxfluff on Wednesday January 19, 2022 @11:09AM (#29995503)
Alright, yea, if you're working under constraints like that, C sucks more than it has to.

It just doesn't feel fair to criticize C without mentioning that your experience comes from working under such unusual constraints. "Strings are hard, C sucks" is quite different from "strings are hard, C sucks when you can't rely on libraries." Also feels unfair to say you get crickets when you ask for a lib but actually you weren't even willing to use one. (There are tons of string libraries for C, it's impossible to miss them if you look around.)

Even then, if you're stuck working with only libc and your own code, there's a very high chance you're doing something wrong if you're doing math on strings and terminating them manually. There's a fair selection of libc functions that do all the math for you and will always output a properly terminated string if your inputs are a buffer & valid size and strings.

Re:BCHS: OpenBSD, C, httpd and SQLite web stack by pull_my_finger on Wednesday January 19, 2022 @09:24AM (#29993916)
Can't vouch for any in particular, but they do exist. https://github.com/oz123/awesome-c#string-manipulation