In principle you're right, but the notion that header file exposes just the "interface" is completely false. Class definition, private variables and functions, etc. are all exposed in header file.

Header files are not a way to only expose the interface. You give up a lot more in C++.

I've never had to deal with pesky header files until I started developing C and it immediately struct me as a royal pain in the ass. Even after couple of years of developing C/C++, I find the whole concept of header files archaic. Include preprocessor directive literally copy pastes stuff with no intelligence what-so-ever. The user is now burdened to ensure #includes are guarded to what I call a patchy half-baked hacked up solution - #IFDEF/#DEFINE/#ENDIF and #pragma in C++.

It should be handled automatically by the compiler/preprocessor or IDE and I believe it is now being addressed in the C++17/20 spec with the advent of "modules". This thing should have been written up way back in 1989.

> In principle you're right, but the notion that header file exposes just the "interface" is completely false.

Sorry, but it is not "completely false". Doing it properly requires a carefully designed interface to hide internal data structures, and splitting out the end user headers from the internal headers, but it works.

> I've never had to deal with pesky header files until I started developing C and it immediately struct me as a royal pain in the ass.

It's like saying, "I've never had to deal with pesky .py files until I started developing Python."

> Doing it properly requires a carefully designed interface to hide internal data structures, and splitting out the end user headers from the internal headers, but it works.

Out of curiosity, how? (If pointers or other mechanisms for memory indirection are allowed then it's pretty easy, so let's agree to ban those.)

Not a rhetorical question, despite the parenthetical.

For global or namespace functions you can hide the implementation in the .cpp file.

Doing the same with objects require pointers to hide the details of any private members. A common pattern is called Pointer to IMPLemenation (PIMPL).

Not sure why we need to exclude pointers?

In c++14 this is very much simplified with the use of unique pointers.

std::unique_ptr<Foo> make_foo(...);

Because I want full encapsulation without giving up by-value semantics. I want my clients to be able to take objects of some opaque record type that I've defined, and put them entirely on the stack, or in some contiguous block of memory entirely of their making, without ever getting to know the constituent fields of the record. I want it all, I want to have my cake and eat it too.

When the parent (er, great grandparent?) says their parent is wrong to deny that headers only reveal the interface, I think it's a little disingenuous to base that on an unrevealed assumption that PImpl is in play. PImpl is basically the the idiom that begot Java. One reason I might opt for C++ over Java, is a desire for finer control over the location of memory -- but it's important for me to know that to actually get that, I'll probably have to sacrifice information hiding. It's a trade-off. Yes, on some level it's better to have the option to make that trade-off, but the product here is "encapsulation or value semantics", not "encapsulation and value semantics".

Mind, I'm not a C++ developer. Maybe these days link-time heroic optimization makes the "right" decisions and collapses these kinds of indirections in all the sorts of situations you'd want it to. I write a lot more Java, and my understanding is that HotSpot gets up to a lot of heroics pertaining to this stuff these days -- I've noticed HotSpot will churn through a workload involving processing a collection of records far more quickly if you can arrange for it to stream through an array, even if you'd expect the records to be scattered randomly throughout memory.

Well, there is a way to have them on the stack even with PImpl: http://www.gotw.ca/gotw/028.htm

However I'd say it's an ugly hack which should only ever be used if you _really_ need the performance.

> I want it all, I want to have my cake and eat it too.

Conceptually this should be possible with some preprocessor/header tinkering; something like:

private.h:

  struct Bar {
    double x;
    double y;
  };
  #define BAR_DEFINED

public.h:

  #ifndef BAR_DEFINED
  struct Bar {
    char opaque[16];
  };
  #endif

  struct Foo {
    Bar bar;
  };

consumer.cpp:

  #include "public.h"

private_implementation.cpp:

  #include "private.h"
  #include "public.h"

this code can crash on platforms where unaligned access is not allowed. you need an alignas on public Bar. and of course the catastrophic bugs if the size of opaque is not kept in sync properly. the point is: the module system should be doing something like this for you behind the scenes.

> I want my clients to be able to take objects of some opaque record type that I've defined, and put them entirely on the stack

if you want to put the object on the stack, the compiler has to know the size of the object to reserve enough space on the stack. How can it know the size of the object if it does not have its full definition somewhere ?

Module symbol table for example, where only the compiler can actually see the complete information about a type, although the consumer code can only access what is exposed as public.

This is fine if you can recompile a program against a new version of the library, but if you just want to relink it doesn't work well. This is actually while common when a .so is replaced with a newer version in a system without rebulding the world.

Some languages have, like ADA I think, have first class support for runtime sized, stack allocated types, so it might work there.

> Not sure why we need to exclude pointers?

Sometimes you need to. You cannot entirely stack allocate an object that uses PIMPL. Also you cannot allocate an array of such objects compactly in memory.

On the other hand, if you want to be able to evolve the class member variables but still maintain a stable ABI, you need to hide the memory layout, for example with PIMPL. But this is a C++ limitation. For example Objective-C* (and also soon Swift) allows modifying the class layout, adding properties etc, without changing the ABI.

https://en.wikipedia.org/wiki/Objective-C#Non-fragile_instan...

> Sometimes you need to. You cannot entirely stack allocate an object that uses PIMPL.

You actually can, if I'm not misunderstanding: http://www.gotw.ca/gotw/028.htm

They must have some level of indirection in the resulting code to accomplish that like virtual inheritance. There is a price and this can be done in C++ too, but you have to opt into the cost.

> Under the modern runtime, an extra layer of indirection is added to instance variable access,

so it's just syntax sugar for PIMPL.

Actually not, it works a bit differently. In Objective-C 1, there was no indirection and one had to explicitly use the equivalent of PIMPL to hide private members from the header or avoid the fragile base class problem.

In Objective-C 2 the object meta-data contains a table of instance variable offsets. The dynamic linker can modify this table at load time so you can freely add both instance variables and methods to new revisions of a class.

So what is the deal? Well, when the holder object itself is heap allocated, pimpl is inefficient because every access will require dereferencing two pointers. Also you cannot put protected or virtual members in the internal pimpl class (then there would be no point to have those in the first place).

That being said, it is not like Objective-C is some pinnacle of performance - you cannot allocate objects on the heap, and the compiler doesn't perform any devirtualization. So for performance critical code you have to drop down to C or...C++ :)

You can split "secret" data structures into another header, and them simply not give that secret header to the customer. If you give the binary and a header with some structures missing, the user can still use whatever you did give them.

> Out of curiosity, how? (If pointers or other mechanisms for memory indirection are allowed then it's pretty easy, so let's agree to ban those.)

Here's one popular way to do it in C++: https://en.cppreference.com/w/cpp/language/pimpl

I guess my tongue in cheek response to your parenthetical would be to ban the use of hidden internal data structures ;-)

How do you compare .py files with .h files? I am confused as to how you are drawing this analogy.

I mean header files are C and C++ source files, so saying you didn't have to deal with them until using C and C++ is a tautology. Likewise, you won't have to deal with .py files until you're using Python.

> In principle you're right, but the notion that header file exposes just the "interface" is completely false. Class definition, private variables and functions, etc. are all exposed in header file.

If you don't want to expose class internals, just use the PIMPL idiom. It's an extra indirection to protect your own abstraction, so naturally C++ decides to opt for performance by default.

That’s an annoying workaround at best. It’s not necessary in C. It wouldn’t be necessary in C++ if you could declare incomplete class types but still declare their member functions. This would have zero performance impact and improve encapsulation.

> It’s not necessary in C

sorry what ? most C libraries I know hide their implementations behind opaque types such as `typedef void* my_handle_t`.

Right, I'm saying that's not necessary, you can just do this:

    struct my_type;
    struct my_type *my_type_new(void);

This is more common than `typedef void*` in my experience, because it actually provides some minimal amount of type safety.

You can also do this in C++ of course, but you can't use member functions this way. My real complaint here is that the Pimpl idiom in C++ is more cumbersome than a simple forward declaration and free functions, which is available in C.

> Include preprocessor directive literally copy pastes stuff with no intelligence what-so-ever.

Well, it's easy to reason about them. The thinking goes about the translation units.

The real problem is that the actual interface is not enforceable: the symbols in the binaries are just names.

> Headers allow you to ship a binary without the full source code.

100% agree. That said, that doesn't mean that they're the only way to do it, or even the best way.