The Linux Kernel also used another trick - placing data in a special named "section."

The linker's job is to combine all the separate pieces in a "section" (special linker term) and write out one section.

Using a special named section lets you create an array using macros, though using it for function pointers is probably safe and using it for other data types might be pretty fragile. http://www.compsoc.man.ac.uk/~moz/kernelnewbies/documents/in...

I was impressed when I discovered our codebase was using this trick to implement new CLI commands. Each C file would implement the functions and helptext etc for implementing a command, but I couldn't find where each command was being registered in the central parser. Because they had to be registered in some central list, right?...

Turned out the macros I overlooked at the end of each file placed the struct defining the command in a special linker section, and the parser init would go and load all the commands from that section.

Interesting to know that this trick already existed in the Linux kernel.

I'm (perhaps perversely) disappointed we aren't using more of the linker's power in many C codebases.

> I couldn't find where each command was being registered in the central parser.

This is the reason why I do not like such tricks. This obviously hurts maintainability and probably portability. Is the tradeoff really worth it? Plain stupid registration is just a single line. KISS.

"Everyone knows that debugging is twice as hard as writing a program in the first place. So if you're as clever as you can be when you write it, how will you ever debug it?" –Brian Kernighan

Yeah, the linker offers a lot of magic waiting to be discovered.

A primary problem with linker magic is that it is usually very non-portable. Hacks you do with macros may be ugly but at least they work everywhere, unless you used a non-standard behavior.

Linker tricks tend to be platform specific, and even more so that other tricks (eg POSIX platforms share a lot of common behavior, but still tend to have quite different linkers).

I had a minor epiphany a few months ago where I realized the linker can be seen as sort of doing dependency injection for a bunch of object files. In theory, you could have a program that depends on some set of symbols, two object files that each include the symbols, and a linker script that at load time checks some configuration parameter before determining which subsystem to link with. You could even get really fancy and dynamically generate another object file with those symbols that acts as a proxy. And from there you could basically implement Spring for C, which obviously is what everyone wants to do ;)

In theory? LD_PRELOAD is specifically designed for it. For examples, see https://rafalcieslak.wordpress.com/2013/04/02/dynamic-linker...

This is roughly how the Windows CE driver architecture works, with "device manager" as the runtime proxy in the middle: https://msdn.microsoft.com/en-us/library/jj659831.aspx

That is how I used to debug C and C++ memory leaks on UNIX platforms, during the late 90's.

I overrided malloc() and free() with my own memory tracking functions.

On Windows side it wasn't needed, as VS already provided some APIs for it.

That's also how C++ static constructor/destructor calls work --- a section contains entirely function pointers, which the init/uninit code in the RTL calls before and after main().

https://msdn.microsoft.com/en-us/library/bb918180.aspx

Another interesting thing is that it relies on the linker ordering names in alphabetical order.

Yeah, the Macintosh Programmer's Workshop (MPW) did pretty much the same thing (back in 1987 or so). I forget the ordering guarantee, it might have been lexicographic by filename. Global initialization in C++ has been a horrible mess, portability-wise, for literally decades.

It is a mess in any language that allows it.

The initialization order is implementation dependent in module/package initialization blocks.

The only guarantee is that the modules being imported run first than the one being executed.

I believe the JVM specification describes precisely when class static initializers run, so that should not be implementation dependent.

Of course it still won't be deterministic if you load classes in parallel in multiple threads.

It's true the JVM specifies this, but it doesn't mean it's always easy or nice to figure out what's going on.

For example

  class A { static final int x = B.y; static final int y = 1; }
  class B { static final int x = A.y; static final int y = 2; }

will not produce any static initializers and will do what you probably wanted because everything can be turned into a class constant and resolved at compile time. However this

  class A { static final Integer x = B.y; static final Integer y = 1; }
  class B { static final Integer x = A.y; static final Integer y = 2; }

will not work the way you might want because there is no order in which the static initializers can be run that will do everything in the correct order.

No it's not. OCaml gets it right.

By forcing developers to remember which order they should be linked?

I wish so much that linker arrays were part of the c and c++ standards! They are much more flexible than pre-main static constructors, and could be used to implement them at the user level in a less magical way!

You would need a linker script for that. Not very accessible for a "normal" project.

Note that this does not work if you make use of __COUNTER__ anywhere else in the file (directly or indirectly).

That's true, and in the event that you're actually using __COUNTER__ for something you could build something similar out of #include with a header that increments an internal number.

  // COUNTER.h
  #if !defined(COUNTER)
  #define COUNTER 0
  #elif COUNTER == 0
  #undef COUNTER
  #define COUNTER 1
  #elif COUNTER == 1
  #undef COUNTER
  #define COUNTER 2
  ...
  #end if

And then change each invocation of ELEMENT to have a leading #include "COUNTER".

  #include "COUNTER.h"
  ELEMENT(x)

  #include "COUNTER.h"
  ELEMENT(y)

Note that if you take this approach then (as far as I know) you're actually writing standard C!

That's really neat, I hadn't considered that. I will definitely try it out, and might end up using it for the library.

I think the other solutions proposed are better, but:

> Sadly, you can't set the value of an index in an array like that in the top level in C, only inside functions.

This is not exactly true. An assignment is a value, you can do e.g.

int a = b[x] = 0;

(which implies that you can do your array-index assignments at the global level by creating nonsensical/unused variables and initializing them).

There's a simple solution you're missing here:

__attribute__((constructor))

This is what I came to say. Here is a working example that builds an array of function pointers at program initialization time.

https://github.com/kbob/schetoo/blob/master/prim.h#L215

This also works across multiple source files, unlike __COUNTER__.

Yeah, it's a weird one! I discovered the idea while building my own toy unit test library, but I never got around to finishing it.

The biggest downside is that (as far as I know) it's impossible to get this approach to coordinate across translation units, so you're forced to define a main in each test file (via the library's header file?). But I doubt that matters too much in practice.