PS: the original motiviation for the STB single-file libs was actually that Windows doesn't have a default place where dependencies are installed:
I've had so many issues with shared libs on linux in the past, I gave up on it and started to include all the necessary source of any third party library in my projects. Makes building and distributing things much easier.
Just because you have five header files instead of five packages doesn't mean you can use all five headers at the same time. For example, some might require C++03 and some might need C++14 or newer.
Also, if all dependencies are single file with no non-standard dependencies, that means none share dependencies they should, like threading, telemetry, or logging libraries.
Anything in its own compilation unit would be isolated. Anything written using C++03 should compile without too much trouble in a more modern compiler.
Also threading is part of the C++ standard library.
I don't know what you mean by sharing dependencies on telemetry and logging. That sounds like adding questionable complexity to libraries that are simple and modular.
In isolation yes. But if it interacts you get events like the fact that gcc changed std::string between versions (from refcounted to one with small buffer optimisation for C++11 compliance) also introduction of move semantics and rvalue references can change interpretation of the same class declaration between compilers in different modes.
You seem to be talking about major changes to the fundamental set up of a project while not taking a comparably trivial amount of time recompiling compilation units.
How often are you changing major compiler versions and breaking standard library versions that you would have this expectation?
You've never linked to a binary? I have no idea what debian's Qt was build with or libssl or SDL etc were built with but we can link to them all the same.
> How often are you changing major compiler versions and breaking standard library versions that you would have this expectation?
You don't configure Jenkins to build against multiple compilers/versions of compiler? Isn't that super normal?
Can trivially support multiple incompatible versions of the library, nothing needs to be fetched or resolved (once you've got the single file of course), can send and distribute it easily over any channel you want (web, email, free file host, google drive, your own website, etc).
* You have to wait for compilation of the whole library at least once every time you do a build of your program - certainly every time you do a clean build, and potentially even incremental builds if it's header only.
* If the library is header only (many of the linked libraries are) then you you potentially have to pay that compilation cost more than once per compilation of your program - once per every one of your source files that include it.
* Again this is specific to header-only libraries, but to avoid code bloat you'll need to turn on link-time optimisation which is far slower than just allowing the linker to do its job by only compiling definitions into a single object file. (Admittedly LTO is a good idea anyway, but adding a bunch of duplicated symbols is avoidable extra work for it.)
* Some useful libraries are realistically just too big for their authors to write the whole thing in one file (e.g. protobuf, opencv, ... in fact most libraries I use on a regular basis seem to fall into that). They could "release" the library in single-file format, similar to SQLite's amalgam, but then if there are any problems (either a bug in their code or something in your code that makes you want to look at their code) you're now not looking at the original source but some mangled version of it.
* If the library is so large that its interface needs to be split over multiple headers (think Boost or OpenCV) then you're now bang out of luck. Hopefully the library has cleanly-enough separated modules you could potentially release these separately (e.g. OpenCV core, imgproc, imgcodecs, highgui, ...) but then you're essentially back to multi-file libraries.
* Adding a library with a lot of its own transitive depedencies takes proportionally the amount of effort as the number of those dependencies, rather than being handled automatically.
One interesting thing about all of these problems is that they get worse and worse as you need more libraries in your program, or need a larger library for your program. In contrast, using a package manager (I'm thinking particularly vcpkg here) tends to add a one-time cost at the start but allows you to scale your dependency list almost for free.
If you're writing your own library, rather than a application, then there are even more problems with this approach, but I won't quite open that can of worms.
In addition, although the name of the repo is "single_file_libs", it links to many double-file libraries consisting of a pair of .c and .h files. These libraries won't have the issues you are talking about. Developers are well aware of the potential linking problems.
That said, you are right that single/double-file libraries tend to be small. It is hard to work with a file with >10,000 LOCs anyway.
I never claimed to be writing only about his libraries, but instead about single-file libraries in general. Sorry if I wasn't clear about that.
> For his (and many others') libraries, you "instantiate" the implementation in a .c file and declare functions in other source files. This way you only compile the library implementation once.
I tried to make clear which of the bullet points only applied to header-only libraries (again, apologies if it wasn't clear). For example, my objection to compiling the same library multiple times specifically applied only header only libraries. As you say, with two-file (or single .c file, if you like) libraries you only have to compile the library once per build of your program - but as I made clear in a separate point, that is still potentially a pain if you do a clean build (but it could be mitigated by having a separate CMake target, or whatever, that contains all your library files).
> In addition, although the name of the repo is "single_file_libs", it links to many double-file libraries consisting of a pair of .c and .h files. These libraries won't have the issues you are talking about. Developers are well aware of the potential linking problems.
As I just said, I had tried to distinguish those cases. They're less problematic, but still somewhat problematic.
> That said, you are right that single/double-file libraries tend to be small. It is hard to work with a file with >10,000 LOCs anyway.
Yes, and if you only need a small number of small libraries then all my objections are a lot less severe. To spin the conclusion on its head - you can avoid going down the package manager route if you avoid having any large dependencies. For some projects that's acceptable. Personally, I find the pain of working without basic largish libraries like protobuf to be enormous compared to the relatively simple process of using a package manager.
The owner of this repo is a well known figure on game development. His libraries are somewhat widely used and have probably inspired quite a few single-header libraries which adopt similar strategies. My comments are not specific to his libraries, either.
I know. In my original comment I tried to make it clear that some (but not all) of my points applied to these libraries that have some separate compilation. In my second comment I apologised for not making that clear and tried to stress it even harder. I'm not sure what else I can do to make it clearer.
> The owner of this repo ... His libraries
If you just use this guy's libraries, and that works out for you, then all power to you.
But I'm worried about C++ beginners stumbling across this page on Hacker News and thinking, hmm, using C++ libraries is so hard that I need to restrict myself to libraries like these. That is simply a myth - using vcpkg is really not hard at all, and well worth it for all the other libraries it gives you easy access to. Again, if that doesn't matter to you then that's fine, but I want readers to know that the option is available.
By the way, that the vast majority of libraries linked to from this article are not by the author and do not have the separate compilation model you talked about - they're simply header only with inline functions.
Even visual studio's compiler compiles sqlite's 6MB in under a second.
> Again this is specific to header-only libraries, but to avoid code bloat you'll need to turn on link-time optimisation
This is nonsense. This list seems like you are trying to invent problems that aren't there. The vast majority of the time you decide what compilation unit to put the definitions into and that's it.
SQLite is notably pure C, which compiles orders of magnitude faster than C++, or at least certain types of C++.
I've worked on a project - not particularly large - where using precompiled headers reduced compilation time from something like an hour and a half to more like 15 minutes. Admittedly that's a very old crusty laptop, but that's without building the libraries, which are compiled separately! Even on much faster modern machines, building the dependencies is at least an hour, maybe more.
I wonder if our opinions are so strongly at odds because we're simply working in totally different situations to start with. As I hope I've illustrated, compilation time definitely IS an issue for us, but obviously it's not for the projects you're working on - which must surely be C-based or at least C-like C++ code. I would wager more people are in my situation, but to some extent that's irrelevant. The advice to other devs has to be: if you're happy to restrict yourself to small C or C-like libraries then single file libraries are fine, but if you want to take advantage of the full C++ ecosystem be aware that there package managers (again, I'm mainly thinking vcpkg) you can use with only a little initial effort. I think it's dangerous for beginners to see articles like the one linked to here in case it makes them think that using C++ has to be like that.
> > Again this is specific to header-only libraries, but to avoid code bloat you'll need to turn on link-time optimisation
> This is nonsense. ... The vast majority of the time you decide what compilation unit to put the definitions into and that's it.
I said that this paricular objection obly applies to header-only libraries, and that qualification is even in the bit you quoted. I never claimed that there are no single-file libraries that allow separate compilation.
But I do dispute that the "vast majority" allow or require separate compilation (either as a separately supplied .c/.cpp file, or by #defining something before one of the uses of the header). That is the exact opposite of my experience. As an experiment I looked at all the libraries listed under "argv" in the linked article, and 9 of them were header only with no option for separate compilation (Argh!, Clara, CLI11, cmdline, flags, kgflags, linkom, optionparser, ProgramOptions.hxx) while only 1 (parg) allowed separate compilation.
But is it due to 'single file libraries'? What I've seen is that having fewer, fatter compilation units speeds up compilation the same way 'unity builds' do. Instead of having a single monolithic compilation unit, a balance can be used to iterate faster and use multiple cores. Compilation units can be groups of things that are changed more or less frequently.
> But I do dispute that the "vast majority" allow or require separate compilation
It isn't about the library always being made for separate compilation. The reality is that unless you are using something fairly fundamental it will probably only need to be in one compilation unit in the first place.
What does need to be shared are data structures and those ideally have as little logic and dependencies as possible.
> But is it due to 'single file libraries'?
That's a fair point, they're generally not. I just meant that if those libraries were single file, but otherwise unchanged, then the problem would only be worse. I see what you're saying about unity builds, but if you build your dependencies totally separately from the rest of your program then it's not so important exactly how fast they build. I'm imagining a situation where you leave vcpkg to build all your libraries overnight, then you spend the next few weeks iteratively working on your application code (which is how it usually works for us).
> The reality is that unless you are using something fairly fundamental it will probably only need to be in one compilation unit in the first place.
In the projects I work on, which are really quite varied, many (but not all) libraries are used throughout the codebase as vocabulary types. Again, I think this just comes down to very different codebases and uses of libraries for us.
I don't know what 'vocabulary types' are, but I think this is an oversimplification. My guess is that most big programs end up like yours but my point is that it doesn't have to be this way. Single file libraries are not the problem and can actually help, although the real issue is deeper.
I think the problem is actually dependencies and many times single file libraries go to great lengths to not depend on anything else. To keep it as simple as possible, I think classes / data structures get over used and instead of just making an interface to some data, people stuff everything they are doing into some class they have.
Transformations from one type to another put into data structures means that the class now has dependencies on those other types. If those types have transformations then they have dependencies and so on. The extrapolation is that everything depends on everything else and even compilation units that should be tiny end up pulling in large parts of the entire program as source code. Lots of compilation units can then mean compiling huge chunks of the source hundreds of times.
This approach lacks any features that reasonable dependency manager provides, such as providing updates compliant with semver.
One counterpoint to your proposed cause & effect is that some observers think Javascript NPM's packaging convenience enables the explosion of single file dependencies. Example discussion:
What’s happening in C++ is that you’re getting one-file libraries that are not small, because developers will shoehorn their library into a single file rather rather than deal with a way of managing C++ library dependencies.
You might find in C++, for example, a header file with over 10k lines in it and a bunch of preprocessor conditionals. What you might find in JavaScript is a library that contains one or two functions.
Maybe we should a couple of You Tube videos about linkers and stuff.
I literally wrote a compiler but still spend 15/20% of my development time trying to find the right not user-friendly CMake syntax, understanding while my Conan dependency broke overnight, or writing header files that could be automatically generated by the compiler (à la GHC).
As for the headers, C++20 modules, or just use an IDE like Visual Studio and the respective wizards.
Which is precisely why C++20 has modules support.
Further, C++ does have a central package manager - whatever the system distributes. C/C++ programs constantly link against the operating system and thus have to get those system headers from somewhere. This means a universal package manager (like Conan) requires buy-in from OS vendors or at least someone who can manage those packages with conviction, else the package manager doesn't make a lot of sense to use.
This is contrast to Node, Ruby and Python as they are inherently Cross-Platform in nature and are not coupled tightly to the OS.
This is why C/C++, historically, have not had a strong centralized package manager.
Anymore, I see less and less package management being used anyway. Most C/C++ projects I work with either vendor in dependencies or use git submodules (the latter I prefer very much).
This is because, unlike e.g. the Node community, micro-dependencies are generally not worth the effort to bring in.
The usual exception to this sort of structure are codebases that will be distributed via a package manager - in which case, they generally rely on the package manager supplying correct versions of the dependencies.
These header-only libs work OK usually because they are able to be included with trivial linkage (something else you normally don't need to care about in scripting environments). Linkage is something you have to care about regardless of if you're writing C, C++ or machine code and since these languages give you pretty much free reign on the flexibility of all of these parameters, it's hard to generalize everything into a nice package like scripting languages can.
Usually the functions inside header only libraries will be static inline, and will almost certainly increase compilation time (noticeably so if you use the header in many places).
There is work being done to improve this situation, it's not like we're all just sitting here going "yes, we love the way things are and have no idea how to make things better." That's far from the case. Shit just takes time.
Certainly, if anyone finds that things have got desperate enough that they're looking the OP's list of single-file libraries then they should just take the one-time hit of setting their project up to use vcpkg. Once that's done, using a supported library reduces to running `vcpkg install foo`, regardless of how many source files it has (or how many transitive dependencies for that matter).
No, I think there is no other way to see it.
Single files are a modern and unified package management platform. You copy the file and you use it. It does not get any simpler than that.
You are talking as if there was a kind of compromise between some imaginary disadvantages of single files. There are none, and there is no compromise. Single files are alright.
Managing dependencies in C++ is a nightmare, and I'm not even talking about the lack of actual modules inside the language itself. It's so bad that people (incl. me) tend to prefer to copy-paste libraries source code instead of setting up a dependency.
A Python, Ruby, Node or Java project would never have to head toward such a poor solution instead of using their respective package managers. But in C++ we do because the alternative (e.g. Conan) is such a nightmare to use.