Is it, though?
I mean, C is around for around half a century, and C is perhaps the primary language in the FLOSS universe.
Still, why is there no widely adopted idempotent directory tree creation library?
In C++ that functionality is indeed part of the standard library.
Oh. And we’re still lacking any kind of sockets support. There’s this thing I’m hearing about called the Internet that’s going to be big any day now and might need this.
The main reason why all these things haven't really been integrated into the main language yet is because the committee is still experimenting with various approaches to asynchronous programming and scheduling with goals of also integrating these with heterogeneous computing.
Also sockets are arguably an outdated networking model anyway, so people really serious about high-performance networking will be doing their own thing regardless.
A TS is almost nothing in the c++ world. Each implementation will have its own quirks and unreliable cross platform behavior if it’s even available. There are well established filesystem libraries too (heck the STL implementation is almost identical to boost for that reason to make it easier for those people to port and for the library maintainers you basically just wholesale copy an implementation or at least have a reference one to test most things against).
> Also sockets are arguably an outdated networking model anyway, so people really serious about high-performance networking will be doing their own thing regardless.
I’m eager to hear what you mean by this because I’m unaware of any OS APIs that do Networking in any other way. Unless you’re talking about userspace drivers or DPDK, neither of which see serious adoption outside of data centers (if even there). 99.99% of an networking code I’ve observed across mobile and cloud is socket-based.
Regardless, the io_uring interface, which is general-purpose and still based on file descriptors for its initialization, does go out of its way to work around some of the problems with the socket model.
Between libcurl, POCO, or even Qt, does it matter at all that the C++ standard offers "any kind of sockets support"?
Just because Java forced everything under the sun into their standard library, that does not mean that move makes any sense at all. Between Boost and Apache and POCO and others like that, it's high time we stop insisting in this nonsense.
You could easily make the same counter argument against including <filesystem> and yet there’s very real value to having something cross platform out of the box that handles elegantly various pitfalls that people typically fall into.
Given that Java followed that path and already acknowledged that mistake by deprecating the original HTTP client, why do people insist in not learning lessons?
As for HTTP within the standard library, I would probably tend to agree although I’d note that Python does have it in its standard module set and I don’t think there’s an acknowledgement that that’s somehow a mistake. Each language fills a different need so framing it as “people insisting in not learning lessons” isn’t helpful. Not to mention that the best way to learn is to experiment and double check if what was once true remains that way (experts know when and in what way to break rules).
What is available as external library has various levels of support quality.
That is why we have POSIX, the C runtime that ISO C didn't want to rely on, yet almost every C program uses a part of it.
Still it was deemed important enough to spend development resources on the POSIX subsystem to make DoD happy, then SFU and now integrate the Linux kernel.
Common sense anyway would suggest that "almost every" is distinct from "many, but also many not", and the latter is what I think would be a better fit here.
The intention behind my comment (and I thought it was clear without any "semantics" discussion required) was to show that there is no need for the POSIX layer to be included as part of ISO C. POSIX is just a "platform layer" (for all I know), meaning that it is much closer to a "library" than a "runtime". I don't see a need to bundle it tightly with the language. Heck, why did I even bring up Windows as a counterexample, when there are probably a lot of more-or-less ISO C conforming compilers for microcontroller platforms that could never support POSIX.
That excuse isn't credible. I mean, I asked about a libraries that handled that usecase, not a standard library. If there was demand for that use case, and that demand popped up so often that it justified the adoption of a de-facto standard, why isn't there any third-party library specialized in that usecase? I mean, putting together a library takes far less work than presenting a proposal to the C standardization committee, which is open to proposals.
So, why isn't there one? Might it be that there is actually no relevant demand for it?
Anyway I just checked and Glib does seem to have it for example.
There is no size limit on how many features could ship with a library. In fact, we live in an age where packages like rimraff report around 50 million downloads per week.
But the original question, and the point it made,still stand: why isn't there such a library to provide such a feature? Might it be that you're overblowing it's demand and usefulness? Perhaps there is no such thing in the standard library because no one bothers with it, or feels it's missing?
And no, people don't reuse trivial libraries in C, this is not JavaScript.
You do realize that tribalist arguments based on absolutely no technical point whatsoever do not make any sense at all, don't you? I mean, who in their right mind would argue that only nodejs developers need to delete dirs?
And if you believe that putting together a reliable cross-platform library that handles idempotent directory creation then be my guest and just do it. How hard would it be, right?
Node.JS is widely known for having people depend on trivial packages (I vaguely remember some story about half the world being affected because the package doing a trim space went down).
The same is not true for C. Libraries there tend to cover larger domains.
Whether that is caused by technical limitations, culture, or competence of the developers is all irrelevant to that statement.
They really aren't. At all. What exactly are the pain points you experienced with C libraries?
> So most C programs use less of them
This assertion doesn't hold any water. In all the years I worked with C and C++ projects, not even once did I ever heard "let's not add a library, it's hard".
Do you actually have a concrete, real issue in mind?
1. Varying build systems. Does the dependency use cmake? Autoconf? A shell script where you’re instructed to modify some variables specific to your system?
2. The above, but my software is shipping on Linux, windows, and Mac. What’s the best way to get this dependency compiled? Does upstream even support all these OSes?
In all my years of doing C programming, I’ve never seen some add a single function library like is_even and then another like is_odd. But you do see it in languages where it is really easy to add libraries.
unless the library author broke those functions out, then I need to rehost it in my environment. so I usually end up forking any library that I want to use. this isn't all bad - I find some bugs and integrate the build. I adopt the library and develop and understanding of it sufficient to maintain it myself.
contrast that to a language environment where those things - strings and threads and build are all standardized.
Your approach makes perfect sense within that context.
It is the intermediary subshell in system() that is the root of all evil, not composing unix programs together; that is what our OS forefathers intended.
The error: you should never pass unescaped/untrusted input into a subshell.
In Python, it’s the difference between the safe:
def make_dir(path):
subprocess.run([“mkdir”, “-p”, path])
def make_dir_unsafe(path):
subprocess.run(“mkdir -p “ + path, shell=True)
os.makedirs(path) subprocess.run([“mkdir”, “-p”, path])
The snippet breaks for paths starting with a minus, as mkdir will interpret those as another option.
The solution is:
subprocess.run(["mkdir", "-p", "--", path]) def make_dir(path):
subprocess.run('mkdir -p'.split() + [path])I’ve seen bugs where some forgot to add quotes and commas to split some arguments and the program did the wrong thing. Parent’s approach avoids that. However, things like spaces in quoted strings are not correctly handled.
I assume you meant something like subprocess.run(['mkdir', '-p', '--', *path.split()]), which in that case using shlex.split will fix your problems with quoted strings.
- performance. Obviously that depends on the circumstances but if you are working on a function that is used in many places you don't know the circumstances.
- error handling and reporting. Are you going to capture stderr from mkdir? If this fails can you provide a clear indication of the problem to the caller?
- paths. If you run mkdir without specifying an absolute path your program may fail if $path is not what you expect. That can also be a vulnerability.
- dependencies. Now you won't run in a distroless container or other cut down environment
- you lose type safety
Not saying "never shell out" but I've been burned by most of these so I am cautious about it.
Certainly the worst of it, but even if you pass the untrusted data as a discrete arguments you're still vulnerable to parameter injection.
Let's say, for example, that you're running ["find", USER_SUPPLIED_PATH, "-type", "f"]. If the user supplies "-delete" as the path, the resulting behaviour would not be as intended.
Would it be noticeably slower to the point it's measurably relevant, though? Or are we talking about something well within the domain of premature optimization?
I mean,if you're doing IO, how often does creating a dir end up being the performance bottleneck?
When you read the code, you know what the types of a, b and c are. You know whether they're built-in types, and therefore whether those operator expressions are actually function calls.
Nothing is hidden, so long as the programmer doesn't make invalid assumptions based on his experience with other, different languages.
In your case, if a, b, and c are all the same type, then b+c can return a completely different type, and you'll need to track that down before you know what type a+(b+c) will have. And in that case, (a+b)+c can have a different type from a+(b+c). If you've memorized the associativity rules for the language, you'll know what to expect -- but it's not at all obvious from the notation.
And, sure. If you know absolutely everything about the language and everything about the codebase, you can make accurate predictions about the behavior of any given line. But if you're coming into a new codebase, there's no way to know at a glance what a given line of code is going to cost. That's what folks are calling hidden complexity.
Oh -- and thanks to the auto keyword, you don't always know the types of a, b, and c without hunting things down.
Yes, you need to look at the declarations of names to know what the resulting type of expressions involving those names is. I don't see that as being surprising or unusual. The same is also true in C.
std::vector<int> numbers = Func();
In that example, there would always be a move or copy there, though there are scenarios in which it might be elided.
This is, of course, an extreme example. But there are definitely cases where it matters a lot.
Especially because the early examples in the article (making subdirs for something like different packages/release - "project/staging", "project/production") are almost certainly safe to use a subshell for - so much so that I was genuinely wondering why the author wouldn't just call out to "mkdir -p" for that use case.
The error came when someone decided to pass untrusted/user-supplied data to this function (the later example - usernames).
But I think doing the later is a huge mistake either way. Don't pass untrusted & unsanitized user data ANYWHERE that isn't explicitly designed for it.
Who gives a hoot whether that's to a subshell, or a pipe/stream, or to a sql query, or your DOM, or any other of hundred of places that user supplied data isn't safe.
If you ever see an unsafe string usage being passed into a function that expects a safe string, without an accompanying call to sanitise(), that's very wrong (and won't work if the language is strongly typed) and should be recognised/corrected as such immediately.
Having the code inline makes it easy to audit, and prevents problems when a dependency changes 5 years later.
It also trains people to write proper C code. Writing loops and checking errno should be second nature for a C programmer, and doing it often helps you learn.