A deep dive into how linkers work (2008)

https://lwn.net/Articles/276782/

by thunderbong • 2 years ago

271 25 2 years ago

lwn.net

skywal_l• 2 years ago

[2008]

But these articles are gold (no pun intended) so always good to see a refresh on HN front page.

ForOldHack2• 2 years ago

Yes. I had heard about someone who had fixed a bug in a linker, and then I thought just how hard can it be, and then there is this.

*GREAT* explanation.

weinzierl• 2 years ago

This is one of my most favourite article series and had so many eye openers for me. I also think there is no other resource, neither internet nor elsewhere, that has all this information in one place. I really wished Ian made a book out of it.

karma_fountain• 2 years ago

The book Linkers and Loaders by John R. Levine is pretty good.

kibwen• 2 years ago

Is there anything in this article series that's not in Linkers And Loaders?

boffinAudio• 2 years ago

I've printed-to-PDF all 20 chapters and have my own book of it now. My only desire is that Ian made a single-page-with-all-chapters view of it available...

perchlorate2• 2 years ago

Someone linked to a Calibre recipe that assembles everything into one ebook. Here is the result if anyone else needs it:

https://www.mediafire.com/folder/b8fdqx7eqcpdl/linker

https://0x0.st/Xycy.azw3

https://0x0.st/Xyct.epub

https://0x0.st/Xycv.mobi

https://0x0.st/Xycw.pdf

fruffy2• 2 years ago

Thanks. It looks like this includes comments which blows up the page count dramatically. A little unfortunate.

perchlorate2• 2 years ago

It's pretty easy to remove them by adding one line to the recipe and re-fetching:

https://www.mediafire.com/folder/u84s3art26lni/no-comments

ForOldHack2• 2 years ago

A PDF? Oh! There it is. Thanks.

ignoramous2• 2 years ago

files on archive.org: https://archive.org/details/linkers.-ian.-lance.-taylor

canucker2016• 2 years ago

The dev who worked on the lld and mold linkers turned the dial to 11...

LLD (part of LLVM): https://llvm.org/devmtg/2017-10/slides/Ueyama-lld.pdf

MOLD linker: https://github.com/rui314/mold/blob/main/docs/design.md

Apple released a new linker that's in the same ballpark as mold - see following link for previous discussion - https://news.ycombinator.com/item?id=36218330

haberman• 2 years ago

I always take this as inspiration that the performance quest is rarely over. LLD was specifically designed to be fast, as was Gold which came before it. But Mold blew them both away.

InDubioProRubio• 2 years ago

https://www.airs.com/blog/archives/51

Pattermatching on assembler code and rearranging and reusage of sequences..

kuharich• 2 years ago

Past comments: https://news.ycombinator.com/item?id=27445981

xyst• 2 years ago

I can see why linkers were created, especially in a time of constrained memory. But given an abundance of memory in modern systems, are linkers even necessary anymore?

As a matter of fact, aren’t these shared libraries a supply chain attack vector (ie, xz attack that was thwarted earlier this year)?

deckard1• 2 years ago

I know it's fashionable to use flatpak, Docker, etc. but I'd still rather not have 30 instances of Gtk running for every GUI app I decide to run. Consider that we still run on Raspberry Pi, etc.

aren’t these shared libraries a supply chain attack vector

Not any more than the apps themselves. If you're downloading a static binary you don't know what's in it. I don't know why anyone trusts half the Docker images that we all download and use. But we do it anyway.

akvadrako• 2 years ago

I think what you mean when you say instance of Gtk is a copy of the Gtk library in memory?

That's not how flatpak works; identical libraries will share the same file on disk and will only be loaded once, just like non-flatpak apps. And because Gtk is usually part of the runtime most apps will use one of a few versions.

citrin_ru• 2 years ago

1. static linking is still linking and you still need linkers to combine multiple object files into a single executable 2. mindset that memory and CPU are in abundance IMHO one of the reasons that user experience is not visibly improving over the years despite orders of magnitude faster hardware

ignoramous2• 2 years ago

But given an abundance of memory in modern systems, are linkers even necessary anymore?

This sudden abundance in memory has been adequately matched by sandboxes, packagers, libraries, and frameworks.

compiler-guy• 2 years ago

Somehow the compiler needs to either have the whole program in one single go--every last source file at the same time, all with exactly the same build options--or there needs to be a way to combine the results of multiple compilation steps.

Even with modern LTO, the compiler doesn't typically see _all_ files in the program at the source code level. Just many. Usually the C-library and C++ library are different.

So as long as various languages don't build the entire program in a single compilation and assembly step, we will need something that combines the results.

That's the linker.

Even building everything statically doesn't eliminate the need for the runtime linker, unless one hard-codes the exact address where a program can run. That runs counter to security measures like ASLR.

account42• 2 years ago

Even building everything statically doesn't eliminate the need for the runtime linker, unless one hard-codes the exact address where a program can run. That runs counter to security measures like ASLR.

You could have the program be position independent (use only relative adressing) and do without a linker for that limited use case.

jcranmer• 2 years ago

Do you want builds to change a single line of code to complete in less than half an hour? If yes, then you need something in the vein of a linker to handle less-than-whole-program units of compiled code.

A deep dive into how linkers work (2008)

Related Stories