Re:A 20-part deep dive into how linkers work
by weinzierl on Thursday August 22, 2024 @03:48AM
(#41317785)
This is one of my most favourite article series and had so many eye openers for me. I also think there is no other resource, neither internet nor elsewhere, that has all this information in one place. I really wished Ian made a book out of it.
Re:A 20-part deep dive into how linkers work
by KerrAvon on Thursday August 22, 2024 @07:03PM
(#41325029)
This is much closer to the metal of a modern ELF implementation. Linkers and Loaders is excellent for background and covers a wider spectrum, but was something like a decade old at the time this was written.
Re:A 20-part deep dive into how linkers work
by boffinAudio on Thursday August 22, 2024 @05:27AM
(#41318304)
I've printed-to-PDF all 20 chapters and have my own book of it now. My only desire is that Ian made a single-page-with-all-chapters view of it available...
Re:A 20-part deep dive into how linkers work
by haberman on Thursday August 22, 2024 @12:41PM
(#41322042)
I always take this as inspiration that the performance quest is rarely over. LLD was specifically designed to be fast, as was Gold which came before it. But Mold blew them both away.
Re:A 20-part deep dive into how linkers work
by xyst on Thursday August 22, 2024 @12:49PM
(#41322113)
I can see why linkers were created, especially in a time of constrained memory. But given an abundance of memory in modern systems, are linkers even necessary anymore?
As a matter of fact, aren’t these shared libraries a supply chain attack vector (ie, xz attack that was thwarted earlier this year)?
Re:A 20-part deep dive into how linkers work
by compiler-guy on Thursday August 22, 2024 @03:58PM
(#41323882)
Somehow the compiler needs to either have the whole program in one single go--every last source file at the same time, all with exactly the same build options--or there needs to be a way to combine the results of multiple compilation steps.
Even with modern LTO, the compiler doesn't typically see _all_ files in the program at the source code level. Just many. Usually the C-library and C++ library are different.
So as long as various languages don't build the entire program in a single compilation and assembly step, we will need something that combines the results.
That's the linker.
Even building everything statically doesn't eliminate the need for the runtime linker, unless one hard-codes the exact address where a program can run.
That runs counter to security measures like ASLR.
Re:A 20-part deep dive into how linkers work
by account42 on Friday August 23, 2024 @06:07AM
(#41327692)
> Even building everything statically doesn't eliminate the need for the runtime linker, unless one hard-codes the exact address where a program can run. That runs counter to security measures like ASLR.
You could have the program be position independent (use only relative adressing) and do without a linker for that limited use case.
Re:A 20-part deep dive into how linkers work
by deckard1 on Thursday August 22, 2024 @01:11PM
(#41322308)
I know it's fashionable to use flatpak, Docker, etc. but I'd still rather not have 30 instances of Gtk running for every GUI app I decide to run. Consider that we still run on Raspberry Pi, etc.
> aren’t these shared libraries a supply chain attack vector
Not any more than the apps themselves. If you're downloading a static binary you don't know what's in it. I don't know why anyone trusts half the Docker images that we all download and use. But we do it anyway.
Re:A 20-part deep dive into how linkers work
by akvadrako on Thursday August 22, 2024 @05:43PM
(#41324624)
I think what you mean when you say instance of Gtk is a copy of the Gtk library in memory?
That's not how flatpak works; identical libraries will share the same file on disk and will only be loaded once, just like non-flatpak apps. And because Gtk is usually part of the runtime most apps will use one of a few versions.
Re:A 20-part deep dive into how linkers work
by citrin_ru on Thursday August 22, 2024 @01:37PM
(#41322604)
1. static linking is still linking and you still need linkers to combine multiple object files into a single executable
2. mindset that memory and CPU are in abundance IMHO one of the reasons that user experience is not visibly improving over the years despite orders of magnitude faster hardware
Re:A 20-part deep dive into how linkers work
by jcranmer on Thursday August 22, 2024 @06:48PM
(#41324950)
Do you want builds to change a single line of code to complete in less than half an hour? If yes, then you need something in the vein of a linker to handle less-than-whole-program units of compiled code.
But these articles are gold (no pun intended) so always good to see a refresh on HN front page.
*GREAT* explanation.
https://www.mediafire.com/folder/b8fdqx7eqcpdl/linker
or
https://0x0.st/Xycy.azw3
https://0x0.st/Xyct.epub
https://0x0.st/Xycv.mobi
https://0x0.st/Xycw.pdf
https://www.mediafire.com/folder/u84s3art26lni/no-comments
LLD (part of LLVM): https://llvm.org/devmtg/2017-10/slides/Ueyama-lld.pdf
MOLD linker: https://github.com/rui314/mold/blob/main/docs/design.md
Apple released a new linker that's in the same ballpark as mold - see following link for previous discussion - https://news.ycombinator.com/item?id=36218330
Pattermatching on assembler code and rearranging and reusage of sequences..
As a matter of fact, aren’t these shared libraries a supply chain attack vector (ie, xz attack that was thwarted earlier this year)?
Even with modern LTO, the compiler doesn't typically see _all_ files in the program at the source code level. Just many. Usually the C-library and C++ library are different.
So as long as various languages don't build the entire program in a single compilation and assembly step, we will need something that combines the results.
That's the linker.
Even building everything statically doesn't eliminate the need for the runtime linker, unless one hard-codes the exact address where a program can run. That runs counter to security measures like ASLR.
You could have the program be position independent (use only relative adressing) and do without a linker for that limited use case.
> aren’t these shared libraries a supply chain attack vector
Not any more than the apps themselves. If you're downloading a static binary you don't know what's in it. I don't know why anyone trusts half the Docker images that we all download and use. But we do it anyway.
That's not how flatpak works; identical libraries will share the same file on disk and will only be loaded once, just like non-flatpak apps. And because Gtk is usually part of the runtime most apps will use one of a few versions.
This sudden abundance in memory has been adequately matched by sandboxes, packagers, libraries, and frameworks.