The problem is of course that it's kind of a fairy tale. The compiler (or assembler) can generate object code, but it needs placeholders for function calls obviously. The problem is, depending on where exactly the called function ends up, the function call might be more or fewer instructions long. That means, depending on what exact kind of call sequence is required, all the other code in the function has to be shifted around, since all jump targets shift around. Even worse, the length of the instruction sequence necessary for a relative jump depends on how far the jump is, so all intra-function jumps may be longer or shorter based on which instruction sequence is required for the inter-function jumps.

Linking sounds like a clean solution, but it really, really isn't. It's messy as all hell.

I think maybe, if I was to design a solution from scratch, I'd make object files basically a binary encoding of assembly code rather than true machine code; notably, I would keep jumps as abstract "jump to this label" instructions rather than proper machine code. This would put the assembler in the linker, where we actually have enough information to do instruction selection, rather than in the assembler where we don't.

We could also just put straight-up compiler IR in the object files, which puts a whole compiler back-end in the linker. This opens the door for LTO. Obviously, there are performance implications of this; if most of the hard work happens in the linker, linking can't be a stupid single-threaded serial operation anymore. But we could just make multi-threaded linkers and we get back our parallelism. We do lose some of the incrementalness of incremental compilation though.