News for hackers. Stuff that matters.
Re:32-bit x86 Position Independent Code – It's That Bad by bandrami on Wednesday April 15, 2015 @01:15AM (#9379013)
The more I play around with musl (the author's C library) the more I'm convinced dynamic linking was not worth the trouble.
Re:32-bit x86 Position Independent Code – It's That Bad by pilif on Wednesday April 15, 2015 @01:30AM (#9379053)
As a programmer who unfortunately is suckered into doing devops at times, the more I get to just update the openssl library instead of the whole OS whenever a security flaw in openssl is announced, the more I'm convinced that dynamic linking was very well worth the trouble.

Funny how perceptions change depending on the angle you're looking at a problem.

Re:32-bit x86 Position Independent Code – It's That Bad by bandrami on Wednesday April 15, 2015 @01:44AM (#9379081)
Oddly enough I was thinking about openssl updates specifically. Did you remember to restart all your long-running clients? (I'm a sysadmin who occasionally gets roped into doing development...)

Dynamic linking makes security more difficult to reason about, which is a Bad Thing. It has many pluses, but also many minuses. And with symbol versioning it gets very problematic to actually figure out what your real code path is to begin with.

Re:32-bit x86 Position Independent Code – It's That Bad by yuriks on Wednesday April 15, 2015 @02:09AM (#9379162)
And when you update all your clients with static linked OpenSSL, did you also remember to restart every single one of them? These seem like equivalent problems to me.
Re:32-bit x86 Position Independent Code – It's That Bad by deathanatos on Wednesday April 15, 2015 @02:47AM (#9379257)
Not quite, I think. With a .so, I can ask `lsof` which services on the machine require restart: the ones that haven't are linked to a deleted so.

And you wouldn't need to restart the clients with static OpenSSL — you need to recompile them. And with static linking, I'm not sure how you would easily determine the linked version out to say, the minor or the micro. (Perhaps, if this is your OS's thing like nix, the package manager keeps track…)

Re:32-bit x86 Position Independent Code – It's That Bad by danieldk on Wednesday April 15, 2015 @02:56AM (#9379291)
Tip for Debian/Ubuntu users: checkrestart from the debian-goodies package is a nice wrapper around lsof that lists running binaries that rely on outdated solibs:

http://manpages.debian.org/cgi-bin/man.cgi?query=checkrestar...

https://gehrcke.de/2014/06/good-to-know-checkrestart-from-de...

It not only shows processes that run older solibs, but in the case of services, it will also give you the commands to restart them :).

Re:32-bit x86 Position Independent Code – It's That Bad by tankenmate on Wednesday April 15, 2015 @06:58AM (#9380063)
It also shows programs that are still using the old binary that has been deleted (unlinked from the filesystem) but not refcount 0 freed. This can also be used to uncover some forms of hiding that hack attempts use.
Re:32-bit x86 Position Independent Code – It's That Bad by hyc_symas on Wednesday April 15, 2015 @03:50PM (#9383595)
Not quite, but you've identified a key leverage point.

You only need to relink the apps, unless the newly patched library breaks its own ABI. Otherwise, it would be ideal for package distributors and package management systems to ship just a single .o file for an app, and do the final linking at package install time. Then updating a buggy static library doesn't require full recompile of any apps.

Re:32-bit x86 Position Independent Code – It's That Bad by pmahoney on Wednesday April 15, 2015 @10:36AM (#9381218)
I don't think that nix would be able to tell you which version of OpenSSL some binary linked statically, as there would no longer be a reference from that binary to /nix/store/xxx-openssl-1.0/lib/libssl.so.

Nix generally uses dynamic linking, though the links are to absolute paths to a specific library version, so upgrading OpenSSL requires a recompile much like with static linking.

With "normal" dynamic linking, upgrading OpenSSL and restarting some service means you're now running untested code, hoping that the new version of the dynamic lib really doesn't change an interface on which your code depends. But to be fair, this is usually a good assumption.

Re:32-bit x86 Position Independent Code – It's That Bad by nimrody on Wednesday April 15, 2015 @09:45AM (#9380890)
Should be relatively easy to make a tool that uses `losf` or similar to get a list of all running processes that depend on a shared library.
Re:32-bit x86 Position Independent Code – It's That Bad by derekp7 on Wednesday April 15, 2015 @12:14PM (#9381965)
I personally would like to see a distribution have a mixed policy -- dynamic linking against core libraries that are part of the default / basic OS install, and static linking against low usage libraries. That way, you keep the "update one library package to fix a bunch of programs", yet you also reduce cross dependency issues by limiting the number of library packages a given program is dependant on.
Re:32-bit x86 Position Independent Code – It's That Bad by bandrami on Thursday April 16, 2015 @05:12AM (#9386720)
IIRC Snowflake[0] does that (but also has a very experimental per-process view of /usr -- pretty cool stuff, really, but takes some getting used to).

[0]: https://github.com/GregorR/snowflake

Re:32-bit x86 Position Independent Code – It's That Bad by edgan on Wednesday April 15, 2015 @01:39AM (#9379070)
You are right. Lack of it will be the next wave of security problems
Re:32-bit x86 Position Independent Code – It's That Bad by zzzcpan on Wednesday April 15, 2015 @07:34AM (#9380200)
There are no problems with updates anymore. You don't have to download everything to update a bunch of static binaries. There are things, like bsdiff, that allow you to download and patch only tiny differences related to a security flaw.

There is no need for dynamic linking for updates. Really.

Re:32-bit x86 Position Independent Code – It's That Bad by _yosefk on Wednesday April 15, 2015 @07:40AM (#9380226)
What if you have binaries compiled from proprietary code bases from umpteen vendors using different compilers and compiler options and what-not, and they all use the Internet for self-updating and stuff and they all need to be patched? How are you going to do that?

Also - weakly related - what if you want to use a plugin from vendor A inside the program of vendor B, and they live in the same address space - how are you doing it without dynamic loading?

Re:32-bit x86 Position Independent Code – It's That Bad by FreezerburnV on Wednesday April 15, 2015 @09:03AM (#9380652)
Chances are, if you're using those binaries, they have their own versions of whatever libraries they depend on already baked into themselves somehow. So even if they depend on OpenSSL, and you update OpenSSL on your system, they're probably still going to end up using a version from a year ago regardless of what you do.
Re:32-bit x86 Position Independent Code – It's That Bad by kryptiskt on Wednesday April 15, 2015 @09:54AM (#9380943)
> What if you have binaries compiled from proprietary code bases from umpteen vendors using different compilers and compiler options and what-not, and they all use the Internet for self-updating and stuff and they all need to be patched? How are you going to do that?

The proprietary code isn't going to chance using system libs beyond libc, because of portability. So they likely still need to be patched separately.

Re:32-bit x86 Position Independent Code – It's That Bad by wtbob on Wednesday April 15, 2015 @08:04AM (#9380325)
> What if you have binaries compiled from proprietary code bases…

Then don't do that. Seriously, RMS, ESR and others have written a plethora of well-reasoned essays indicating why proprietary software is a poor choice.

Re:32-bit x86 Position Independent Code – It's That Bad by eropple on Wednesday April 15, 2015 @09:52AM (#9380930)
So you're admitting that this isn't a solution and you need to change the rules of the game to make it work.

Well, I'm convinced.

Re:32-bit x86 Position Independent Code – It's That Bad by wtbob on Thursday April 16, 2015 @07:06AM (#9387068)
'Doctor, it hurts when I stab myself in the eye!'

'Well, don't stab yourself in the eye.'

Re:32-bit x86 Position Independent Code – It's That Bad by GFK_of_xmaspast on Wednesday April 15, 2015 @09:15AM (#9380713)
ESR advocating for something is reason enough to seriously consider the alternative.
Re:32-bit x86 Position Independent Code – It's That Bad by zzzcpan on Wednesday April 15, 2015 @07:52AM (#9380260)
I don't see a problem with proprietary code bases.
Re:32-bit x86 Position Independent Code – It's That Bad by q3k on Wednesday April 15, 2015 @06:12AM (#9379933)
There is however, an important security benefit from PIC: ASLR. It's not a silver bullet, but can stop a range of vulnerabilities from being reliably exploitable usable unless an memory leak is also available.

Even if you so far as control the instruction pointer as an attacker, you might just not know where to jump to in the target address space...

Re:32-bit x86 Position Independent Code – It's That Bad by hyc_symas on Wednesday April 15, 2015 @03:54PM (#9383615)
ASLR is a pretty hackish solution to the main problem, which is that the function call stack and parameter stack are combined. The problems that ASLR mitigates don't exist in languages like Forth, where there are two separate stacks - a call stack and a separate parameter stack.
Re:32-bit x86 Position Independent Code – It's That Bad by StillBored on Wednesday April 15, 2015 @05:03PM (#9384105)
This is really an ABI issue, more than a language one. I'm not sure anything in C requires a combined stack. Plus, processor arches like ARM and POWER use link registers rather than a stack to store the return address. Its the ABI that then chooses to put the LR on the parameter stack. Even on x86 SP could be dedicated as a function stack and separated from BP (or pick another x86_64 register) as the parameter stack.

Of course being an ABI issue, probably makes it really hard to solve, and hence hacky solutions abound. (actually its probably pretty easy to solve given LLVM, just impossible to get anyone to accept).

Re:32-bit x86 Position Independent Code – It's That Bad by hyc_symas on Wednesday April 15, 2015 @11:28PM (#9385743)
Hm, you may be right that nothing in C requires it. But things like POSIX pthreads do - e.g. pthread_attr_setstack() only lets you set a single stack address and size.

So yes, you'd have to create a new runtime from the ground up, not just a new ABI for an existing API.

Re:32-bit x86 Position Independent Code – It's That Bad by q3k on Wednesday April 22, 2015 @08:48PM (#9424428)
ARM does store the return PC in a link register, but unless you're in a leaf function, it will still be pushed into the stack and can be overwritten by overflows.
Re:32-bit x86 Position Independent Code – It's That Bad by q3k on Wednesday April 22, 2015 @08:47PM (#9424427)
It's foolish to think ASLR only prevents stack smashes - it also prevents unlink-style attacks via heap overflows and use-after-frees, data overflows into pointers and other segments, and many other vectors.

Yes, it is a mitigation strategy that wouldn't exist in an ideal world. But we have to do best with what we have, and security takes a very pragmatic approach at this.

Re:32-bit x86 Position Independent Code – It's That Bad by justincormack on Wednesday April 15, 2015 @08:27AM (#9380464)
OpenBSD now has ASLR for static executables. There was some discussion of it on #musl so it might happen.
Re:32-bit x86 Position Independent Code – It's That Bad by deweerdt on Wednesday April 15, 2015 @01:47AM (#9379088)
One of the main reasons for dynamic linking has become irrelevant, I believe: the availability of disk and memory space has grown faster than the size of the binary objects.
Re:32-bit x86 Position Independent Code – It's That Bad by pilif on Wednesday April 15, 2015 @02:38AM (#9379236)
IMHO, static linking, be it using a mechanism alike what npm is doing or be it plain-old static linking of binaries also means "you link it, you own it".

For every package, you link statically, you as the parent package owner become responsible for all security flaws of all the packages you link statically.

And by "responsible" I mean: Every dependent security announcement of a dependent package also becomes your security announcement.

If you're AWESOMY 1.1 and you link statically against openssl and openssl announces a security flaw, then you better and quickly release AWESOMY 1.1.1 with an accompanying security announcement too.

Are you willing to do this? Do you trust the chain of dependencies all the way down to also be willing to do the same?

As a responsible developer, I'd much rather delegate that responsibility away to a packager or even the user, especially with well-known libraries like openssl.

If I'm owning AWESOMY 1.1 and link dynamically against openssl, then I don't have to do anything when openssl releases a security announcement. I can, if I want to, inform my users to maybe update openssl, but with some likelihood they are already doing this anyways for some other package.

For me as a developer, this is considerably more convenient.

Yes. Static linking has huge advantages for me as a developer too, but it also comes with a great many additional responsibilities I'm personally not willing to take on, also because I don't trust my dependencies to be as diligent about their dependencies.

Re:32-bit x86 Position Independent Code – It's That Bad by roghummal on Wednesday April 15, 2015 @05:15AM (#9379754)
At least on Linux, (commercial) people are packaging shared libs with their binary and using LD_PRELOAD wrappers anyway. "Own-it-by-proxy"-except-you're-shipping-it-so-you-kinda-own-it-anyway.

Dynamically linked, dlopen()-or-equivalent plugin designs are an exception. Otherwise, I'd like to see more statically-linked applications.

Re:32-bit x86 Position Independent Code – It's That Bad by bandrami on Wednesday April 15, 2015 @04:08AM (#9379497)
For every package, you link statically, you as the parent package owner become responsible for all security flaws of all the packages you link statically

Well, speaking as an administrator rather than a developer: a developer shouldn't be making that kind of policy decision (time of symbol resolution) to begin with barring a strong technical need (plug-in based architecture, etc.).

I mean, 90% of packages don't care whether their libraries are dynamic or static, but the ones that do can still be annoying. (Coreutils, of all things, requires dynamic linkage for stdbuf, though that seems to be considered a bug by the maintainer; on the other end of the spectrum, getting Perl to even build statically is like pulling teeth, and that was a design decision.)

Do you trust the chain of dependencies all the way down to also be willing to do the same?

The only time that is a problem is those cases where the package developer literally plunks down a bunch of code from his upstream into his own tree (think the embedded glib inside pkg-config, or the hideous monstrosity that is gnulib). I think all sane people agree this is bad -- if you aren't significantly altering the code (at which point it's "yours") there's no sense in doing that and risking a sync problem. But that's not even static linkage; that's just literal code-sharing.

Re:32-bit x86 Position Independent Code – It's That Bad by vetinari on Wednesday April 15, 2015 @07:52AM (#9380257)
The embedded glib inside pkg-config solves the chicken-egg problem: pkg-config requires glib and glib requires pkg-config. This way, you can build pkg-config first with its embedded glib, then glib itself and then go solve your own problems, instead of butchering the build system to make them build at all.
Re:32-bit x86 Position Independent Code – It's That Bad by bandrami on Thursday April 16, 2015 @05:14AM (#9386725)
Oh, true, and I didn't mean that as a criticism of pkg-config; they only did that because it was literally the only option. In general you should only duplicate actual code if it's the only option, was my point; pkg-config is the pattern and gnulib is the anti-pattern in that.
Re:32-bit x86 Position Independent Code – It's That Bad by deweerdt on Wednesday April 15, 2015 @02:49AM (#9379261)
I believe that what you describe is the remaining argument in favor of dynamic linking.

That said, I'd like to point out that it is convenient for the developer - since customers are ultimately interested to know if your software is vulnerable, and you'll have to explain how the vulnerability affects it -.

It's also a double edged sword: openssl has a good ascending compatibility record, but that can't be said about all user space libraries. IOW, it's tricky to guarantee that your software will work flawlessly across all the incarnations of CentOS 6.x, if you have a lot of external dependencies, for example.

Re:32-bit x86 Position Independent Code – It's That Bad by danieldk on Wednesday April 15, 2015 @03:10AM (#9379335)
IOW, it's tricky to guarantee that your software will work flawlessly across all the incarnations of CentOS 6.x,

It's not so tricky, since Red Hat specifies nowadays what guarantees can be expected for what packages:

https://access.redhat.com/articles/rhel-abi-compatibility#Ap...

Re:32-bit x86 Position Independent Code – It's That Bad by pjc50 on Wednesday April 15, 2015 @04:36AM (#9379590)
Dynamic linking is required for anything resembling a plugin architecture (such as PAM).
Re:32-bit x86 Position Independent Code – It's That Bad by wtbob on Wednesday April 15, 2015 @08:06AM (#9380342)
Not really—one could use processes and some form of IPC (shared memory, pipes, whatever). The efficiency could be pretty bad, but the safety and reliability could be better.
Re:32-bit x86 Position Independent Code – It's That Bad by JoachimSchipper on Wednesday April 15, 2015 @09:52AM (#9380934)
In fact, OpenBSD delegates logins to a login_<foo> binary, e.g. http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man8/.... That works. OpenBSD's system is not as flexible as PAM, with the attendant upsides and downsides.
Re:32-bit x86 Position Independent Code – It's That Bad by bandrami on Wednesday April 15, 2015 @12:39PM (#9382199)
Yeah. There's a reason I stick with OpenBSD.

I have absolutely no need for my authentication system to be "flexible".

Re:32-bit x86 Position Independent Code – It's That Bad by StillBored on Wednesday April 15, 2015 @03:17PM (#9383406)
"And by "responsible" I mean: Every dependent security announcement of a dependent package also becomes your security announcement."

Yes, and no, because as the developer you can look at the security vulnerability and decide if its actually exploitable in your application. That is assuming you can determine it, but in a lot of cases its simple. Especially in a huge library like openSSL. Say for example the only thing i'm using openSSL for is some limited functionality, say SHA256, then I probably can ignore 99.99% of the security issues because they just won't apply.

I've been in this situation with an embedded platform that ships as part of the product I work on. Its pretty much got daily security updates, and yet we rarely get hit by any of them because our usage of the platform is like 1% of its functionality.

Re:32-bit x86 Position Independent Code – It's That Bad by zvrba on Wednesday April 15, 2015 @02:50AM (#9379267)
This is an oft-repeated mantra, but is it really true? I don't have a linux desktop running here, but try an experiment: start up a desktop environment such as KDE, and write "free -m" and see how much is there under "shared" heading. Without shared libraries, some extra memory corresponding to some multiple of that number would be used.

To calculate exactly how much memory is saved by shared libraries, you'd need to write a kernel module to walk the internal structures describing physical pages and summing reference counts of used pages. Maybe it's already been done?

Re:32-bit x86 Position Independent Code – It's That Bad by bandrami on Wednesday April 15, 2015 @04:47AM (#9379632)
some extra memory corresponding to some multiple of that number would be used

Depends. If I have 7 instances of my terminal emulator loaded (say I hadn't discovered tmux yet or something), the loader can share their .rodata and .text segments, and in many cases it does (YMMV; heuristics apply; void where prohibited; etc.). So a lot of things that are in shared libraries right now might still be only loaded into memory once if their binaries are segmented correctly.

The Plan9 people (Plan9 doesn't do dynamic linking) claim that the memory savings they get from skipping the relocation overhead are greater than the memory hit from the times that the same stuff does get loaded multiple times, though obviously always take self-promotion with a grain of salt.

Use case probably matters -- my servers run few processes to begin with, and it's often a lot of versions of the same process, whereas my laptop runs a ton of very different processes. Same sort of argument that makes me happy with udev on my laptop while also very happy with a static /dev tree on my servers.

Re:32-bit x86 Position Independent Code – It's That Bad by zvrba on Wednesday April 15, 2015 @05:46AM (#9379849)
I agree, sharing will (most probably) still work on segment-level [0]. But this means that if I have two different executables each linking an identical version of, say QT, in its .text segment, those copies will not be shared.

[0] In fact, certainly. Program loading works by mmaping the executable with MAP_SHARED flag into the process's address space, and the VFS takes care to keep all mappings coherent. The simplest way of keeping mappings coherent is to actually share the backing storage between all mappings. It's the foundation of CoW.

Re:32-bit x86 Position Independent Code – It's That Bad by caf on Wednesday April 15, 2015 @05:04AM (#9379702)
You could do that calculation in userspace with a stock kernel, by summing up the sizes of the non-writable mappings of the various library files in /proc/*/maps, then using mincore() to find out how much of the libraries are resident.
Re:32-bit x86 Position Independent Code – It's That Bad by zvrba on Wednesday April 15, 2015 @05:50AM (#9379867)
mincore is a system call and does not take a pid parameter. This means it has to be executed by each process individually, which would require injecting code into each running process and executing it in some way.

Unless there's a tool which makes this extremely simple (maybe Intel's pin?), I believe that writing a kernel module is simpler. The module's init function tallies up the pages and writes out the result into the kernel log. Then the module exits.

Re:32-bit x86 Position Independent Code – It's That Bad by caf on Wednesday April 15, 2015 @06:17AM (#9379949)
You don't need to run mincore() on target pids - you just need to write a tool that opens(O_RDONLY) and mmaps(PROT_READ) each library file, then calls mincore() on each page in the mapping to find out which pages of the library are loaded shared.

The results of mincore() in one process with a shared mapping of a file are enough to tell you how much of that file is loaded shared system-wide.

Re:32-bit x86 Position Independent Code – It's That Bad by zvrba on Thursday April 16, 2015 @03:06AM (#9386330)
Just found out that physical page information is exposed through /proc: https://www.kernel.org/doc/Documentation/vm/pagemap.txt
Re:32-bit x86 Position Independent Code – It's That Bad by zvrba on Wednesday April 15, 2015 @06:34AM (#9379992)
Ah, good point! Though I think I'll do a dlopen on each file to mimic whatever the loader usually does. I'll do it as a weekend-project :)
Re:32-bit x86 Position Independent Code – It's That Bad by caf on Wednesday April 15, 2015 @06:57AM (#9380058)
In case it helps, here's my little utility (which just shows how much of a file is in core):

https://github.com/keaston/fincore

Re:32-bit x86 Position Independent Code – It's That Bad by heeen on Wednesday April 15, 2015 @05:15AM (#9379753)
Are you suggesting that every single calculator, file manager, terminal, editor, package manager and music player ship with their own copy of libQt5* (around 78M) or gtk?
Re:32-bit x86 Position Independent Code – It's That Bad by bandrami on Wednesday April 15, 2015 @05:43AM (#9379839)
Static linking doesn't pull the entire library; it pulls the parts of the library it uses.

That said, QT and GTK probably should be broken into smaller libraries in a perfect world.

Re:32-bit x86 Position Independent Code – It's That Bad by pdkl95 on Wednesday April 15, 2015 @06:45AM (#9380021)
So I take it you're offering to buy me some more RAM, then? Unfortunately, my motherboard is already full at 4GB so this upgrade would be more expensive than you might expect at firs. What about my friends that have older 2GB laptgops? Do they get an upgrade too?

I'm joking, of course, but there IS a lot of variation in computer hardware, which makes this kind of broad generalization even more problematic.

Also, memory size is probably not that useful of a metric for modern hardware, where the penalty for overflowing the CPU caches can be huge.

edit:

Why statically link when you can prelink(8) instead?

( http://linux.die.net/man/8/prelink )

Re:32-bit x86 Position Independent Code – It's That Bad by simias on Wednesday April 15, 2015 @07:37AM (#9380210)
Can you give more details? Isn't this particular issue pretty much x86 specific anyway? I think it's not a problem anymore on x86_64.