> What is great about header files?

One great thing - it's nice to have an interface[1] which already has tooling to let other languages use your private, hidden and compiled implementation.

If those C++ libraries that almost all of Python scientific work is built on had used modules[2] (or something similar) instead of headers, interfacing Python to those libraries would have been an immense effort.

Even with modules existing now, if you're writing something core and foundational in C++, it's not getting adopted like Numpy, BLAS, etc unless you also provide header files.

And that's just Python. Think of all the other languages that rely on performance critical code written in C++, and used with a C-only interface.

[1] Of course, your interface can't be a C++ interface either, so there's that.

[2] If modules were around at the time.

That's interesting! Why would modules make it harder to interface with c++ from python ? Is it because the header files are a direct interface to the functions? How hard would it be to do the same from modules, would it just require tooling to support modules rather than directly provided header files?

(I just realized that I don't completely understand how exactly the new modules work, so I have some further reading to do! )

> Why would modules make it harder to interface with c++ from python ? Is it because the header files are a direct interface to the functions?

Well, they're simpler, they're text, and the tooling already exists (I use swig all the time to automatically make C functions and symbols available to Java and Python. Some Lisps as well).

The reason the tooling exists, is because they are simple, and they are text. Looking at the C++ module definition, it's unclear to me[1] that similar tooling for C++ modules will ever be possible.

As long as the most popular languages (Python, Java, etc) rely on a C header file for inter-call interoperability, header files aren't going away. No one wants to manually write the interface into a C++ library by hand.

[1] I'm no C++ expert though - the problems I see with modules includes things like lack of a standard to grab the interface from the compiled module, complexity of C++ to determine the interface from the actual source code, etc.

> Well, they're simpler, they're text, and the tooling already exists (I use swig all the time to automatically make C functions and symbols available to Java and Python. Some Lisps as well). > The reason the tooling exists, is because they are simple, and they are text. Looking at the C++ module definition, it's unclear to me[1] that similar tooling for C++ modules will ever be possible.

Perhaps there's something specific about C++'s module design that makes this difficult, but Rust has modules and has similar tooling to enable you to automatically generate bindings to other languages (including Python and Java).

Really there's nothing stopping tooling like swig (or a C++ compiler for that matter) from processing module interfaces as text, just like headers. The language spec for modules doesn't require (or even really talk about) compiling them to any kind of binary or intermediate format.

Rather, the language spec aspect of modules is about symbol visibility and collisions. In terms that interop cares about, moving a declaration into a module just adjusts its name mangling, without changing much else about e.g. its presence in the object file.

> Really there's nothing stopping tooling like swig (or a C++ compiler for that matter) from processing module interfaces as text, just like headers.

Well the one hurdle is that C++ is a lot more complex to process than C.

You'll note in my original comment upthread, I pointed out that the C++ libraries provide a C interface in their headers, and that is what tools like swig work with.

They don't (and can't, because of symbol mangling) read C++-only headers. Since they can't do it for headers (because the resulting symbols differ from compiler to compiler), why would they be able to do it for modules?

I'm of the understanding that C++ modules are unable to provide a C interface (`extern "C"` or similar), so it's kinda pointless reading them and generating a list of symbols, because then the generated stubs only work when the library is compiled with that specific compiler.

I think that it would be nice if the compiler can be instructed to output header files for the modules it reads. You'll still have the problem that the header now has mangled symbols in it (because modules cannot have `extern "C"`), but at least tools can generate stubs for that specific instance of the compiled library.

(PS. I'm happy to be corrected about the `extern "C"` thing - if I am wrong about that then, sure, tooling that scans C++ modules for `extern "C"` can in fact be made quite easily).

You can write `extern "C"` declarations in modules, actually. And either way the complexity of C++ and its name mangling is independent of headers vs modules, no?

Further, swig does seem to support quite a bit of C++. IIUC it even avoids dealing with name mangling by generating wrappers that provide a C interface.

> You can write `extern "C"` declarations in modules, actually.

See, I did not know that :-)

> And either way the complexity of C++ and its name mangling is independent of headers vs modules, no?

You got me self-doubting here: C++ implementations are necessarily more complex than their interfaces.

The full complexity of C++ (not restricted to only name-mangling) is not usually found in headers intended for consumption by a C-interface compatible consumer.

The intention of headers is to serve as an interface alone, but absolutely nothing enforces that - nothing stops the developer from dumping the entire implementation into the header (and this is a popular route for some libraries that are header-only).

Modules are not intended to serve as an interface alone, so it is more likely that devs (myself included) will simply throw the entire implementation into a module, because it seems like a better idea to do so with modules.

IOW, common practice for headers is to only have the interface necessary to use the implementation, but I think that common practice for modules will be to have the implementation and interface included in a single module.

> Further, swig does seem to support quite a bit of C++. IIUC it even avoids dealing with name mangling by generating wrappers that provide a C interface.

Swig supports much of C++ in a smart way, by default. For things like demangling swig generates `extern "C"` wrappers (for functions, definitely - not so sure about class typenames and enum typenames).

But, to generate those wrappers from modules requires the original source code for those modules, and to use generated C++ wrappers, it needs to be compiled with the same compiler: Not a large hurdle, but it is definitely an additional concern compared to using the headers.

It will be interesting to see how this plays out: how long it takes until swig support for modules, or swig-like tooling for modules, comes along. It's still early days for modules, after all.

> Modules are not intended to serve as an interface alone, so it is more likely that devs (myself included) will simply throw the entire implementation into a module, because it seems like a better idea to do so with modules. > > IOW, common practice for headers is to only have the interface necessary to use the implementation, but I think that common practice for modules will be to have the implementation and interface included in a single module.

While I'm sure some people will do this (just like they do with header-only libraries, and are forced to do with C++ features like templates) modules do support the same interface/implementation split as headers.

I suspect that a lot of code will stick with that split for a couple of reasons. First, they already have it with headers, so migrating over to modules will be less work if they just leave things that way. And second, downstream translation units that import the module can't (with today's compilers) even start building until their dependencies' interfaces have finished building, so keeping the split may also turn out to improve build times.

Really though, I think the ideal here is not for library authors to have to maintain C-ish interface headers as a sort of conflation with that interface/implementation split. That should be generated by a tool- either something like swig (which knows more about the other side of the binding) or at least a C++ tool like you mentioned upthread. This is also how, for example, Carbon plans to do its C++ interop.

> But, to generate those wrappers from modules requires the original source code for those modules

This part I don't think is actually going to be an issue, because pre-compiled module interfaces are not really any more usable in C++-land than pre-compiled libraries. They also have to be compiled by the same compiler, with the same configuration, so people will have to provide the module interface sources just like they provide headers.

Headers are a really bad interface IMHO. Yes there’s tooling, but they require your headers be very very specific. Any deviance can break that tooling, so you’re either lucky in what you can bind or are writing to a very strict spec.

Modules on the other hand are more strict and would likely have made the job of writing language bindings easier and more predictable.

I’m not sure I agree with your logic about NumPy etc. You seem to be conflating technology limitations with policy ones. They could most certainly use modules too , since they exist in the compiled space anyway. They won’t because they make certain compiler compatibility guarantees but that’s a conflict of interests not of technical quality.

In general your argument feels a bit “older is better due to legacy support”. That would preclude almost any advancement in the space as well, because any new language feature (even just a keyword) would break all your examples above equally too.

> Modules on the other hand are more strict and would likely have made the job of writing language bindings easier and more predictable.

There's two ways for tooling like swig to work with modules:

1. Read the compiled module and generate the stubs to call into it.

2. Read the source code for the module and generate the stubs for it.

Which of those, do you think, is easier than reading C (not C++) header files?

Reading the module information would be significantly easier. (which is not something swig can do today since swig hasn’t been updated in a while )

Parsing code to define an API is horribly fragile. A single macro or unknown keyword can break it. Hence why you had to specify C not C++ headers.

Having a statically defined metadata like C++ modules do would have greatly simplified binding generation

They're straightforward and easy to reason about.

Textual inclusion also allows a number of things modules do not.

The main disadvantage that they have, that they're stateful, is also true of module systems in many languages (e.g. python).

I greatly disagree with “easier to reason about”

Maybe in a clean, smaller codebase with minimal dependencies.

But the textual inclusion means that including one header can suddenly break previously compiling code. Unless that header is completely inclusive, that can mean a while of chasing down transitive headers till you find the problematic one.

The semantics are literally "it's the same as copy-pasting it where it's used". It doesn't get simpler than this.

There are much more things going on in the module system that you'd have to know about if you care about what's actually going on.

The module system doesn’t have side effects on inclusion, whereas headers do.

So again, maybe if you have a smaller, cleaner or dependency free codebase, then it’s not an issue.

But in any codebase of major scale or legacy, headers can introduce several issues that aren’t possible with the module interface.

The side-effects are merely the rules of C++, that you already know by virtue of writing C++.

I feel like you’re talking past any of the caveats I’ve mentioned twice. I have no interest in belabouring the point with someone who’s clearly ignoring what’s being written. Have a good day.

You're really burying the lede here.

Headers are literal copy paste, and are entirely different. Anyone who has ever had to define WIN32_LEAN_AND_MEAN knows that there's _way_ more going on here.

This exactly, thank you. Sometimes you can’t untangle the defines in another header and they infect your own.

Modules are a nice barrier to guard against side effects.

Yes, the number of times I've had a `LoadLibrary` function name conflict with `windows.h`'s `A`/`W` suffix-switching or had to deal with X11's `#define Status int` "typedef" mucking with completely unrelated code is…sad.

Just don't include those in headers.

That's not possible. Windows.h is a requirement for anyone working on Windows. Various standard library implementations make use of macros to implement debug-only assertions, or bounds checks in containers.

Don't include it in headers = only include it in a .cpp file.

Because `void*` to pass `HANDLE` around is better how? Maybe if you're able to hermetically seal off platform-specific bits, but usually you'll have two parts that want to talk about Windows API types across their call boundaries…

It's called encapsulation.

Sure, but I also prefer talking directly about what you're talking about though. If you have some Windows implementations that need to talk to each other, I'd rather they use HANDLE and HMODULE than "void* winkwinknudgenudge" style things. Hopefully users don't need to include these headers directly (but sometimes they do), but it's still something you need to consider in the APIs and such because while the header might say LoadLibrary, the caller is going to need to include windows.h and deal with the #define anyways.

any interaction with the win32 api should be hidden within your library.

Surely you want your code to be portable to begin with?

It can be portable and still want to expose native information. SDL2 is probably something we can all agree is portable, but this still exists: https://wiki.libsdl.org/SDL2/SDL_SysWMmsg

Separation of interface and implementation. In languages without them you must rely on documentation generators or an IDE to explore a library/packages public interface. Header files provide this separation without the need for fancy tools.

Python added pyi files in after the fact. Adoption on those is much slower than I would like.

why would be using a "fancy tool" a bad thing? Headers are to be maintained manually, which is lost productivity and invite errors. The "fancy tools" can provide specialized search, formatting of the documentation, internal links, formatted and tested code examples, and of course the interface for free. In modern languages they're even part of the included tools.

I'm having a hard time seeing the downside of the "fancy tools".

You immediately see the public interface of a library without the implementation getting in the way (for C headers at least, typical C++ headers are an unreadable mess because they mix interface declarations with inline/template implementation code).