News for hackers. Stuff that matters.
Re:Sorry state of dynamic libraries on Linux by quotemstr on Monday September 17, 2012 @05:37AM (#4531824)
For a long time, I thought that the ELF shared library model was the obviously correct thing, but over the past few years, I've come to realize that Windows got shared libraries right after all.

On Windows, all inter-module dependencies are (module, symbol) pairs (written "module!symbol"), not just bare symbol names as on ELF systems. That is, modA!fun1 calling modB!fun2 can exist in the same process as a modC!fun1 calling modD!fun2. The Windows model doesn't permit global interposition, but that's a good thing: the lack of interposition support permits the optimizations Macieira mentions, but "hooking" is still possible through a variety of mechanisms, which usually involve either overwriting import and export tables or the replacement of function preambles with jumps to trampolines.

Regardless of the performance considerations, I think the Windows DLL approach is the more robust and conceptually lighter one. That the Windows approach is also faster is simply a beneficial side effect.

Re:Sorry state of dynamic libraries on Linux by FooBarWidget on Monday September 17, 2012 @08:18AM (#4532166)
While ELF has its problems, I wouldn't call Windows's approach 'right', by a long shot.

* Writing libraries is a big hassle in Windows because you have to explicitly define all exportable symbols.

* DLLs have all kinds of strange boundary rules. C++ exceptions cannot pass DLL boundaries. Heap memory allocated by one DLL cannot always be freed by another. Etc etc.

Re:Sorry state of dynamic libraries on Linux by shin_lao on Monday September 17, 2012 @09:01AM (#4532322)
The first problem is actually not a problem but one of the reasons DLLs are better: private by default reduce linking problems.

The second problem has got nothing to do with DLLs and everything to do with the fact that there is no C++ ABI.

As for memory allocation, it's not really a good idea to allocate memory in a library and free it in another, is it?

Re:Sorry state of dynamic libraries on Linux by npsimons on Monday September 17, 2012 @11:01AM (#4532854)
As for memory allocation, it's not really a good idea to allocate memory in a library and free it in another, is it?

Ah, I guess I'll have to stop using strdup() then.

Re:Sorry state of dynamic libraries on Linux by shin_lao on Monday September 17, 2012 @11:10AM (#4532895)
strdup(), part of the libc, allocates memory with a function from the libc (malloc), therefore you are allocating and freeing memory with functions within the same library.

However, yes, for security reasons strdup should be avoided.

Re:Sorry state of dynamic libraries on Linux by pjmlp on Monday September 17, 2012 @10:15AM (#4532641)
> Writing libraries is a big hassle in Windows because you have to explicitly define all exportable symbols.

Just like in Aix, unless it is now changed (my experience goes back to 2001).

> C++ exceptions cannot pass DLL boundaries.

C++ restrictions have to do with lack of C++ ABI. You have in same issue in ELF systems, when mixing compilers.

> Heap memory allocated by one DLL cannot always be freed by another.

This is a good thing. Object handles belong to the module that created them, and it is a good way to ensure a clean way to use accessor functions to deal with data.

Re:Sorry state of dynamic libraries on Linux by klodolph on Thursday September 20, 2012 @12:12AM (#4546910)
> This is a good thing. Object handles belong to the module that created them, and it is a good way to ensure a clean way to use accessor functions to deal with data.

No, it doesn't prevent anything. There is nothing stopping you from calling free() on a buffer returned from a function in a DLL, it just might crash. There is nothing inherently superior about adding an additional source of crashes which are not diagnosed by the compiler.

Re:Sorry state of dynamic libraries on Linux by npsimons on Monday September 17, 2012 @11:00AM (#4532849)
C++ exceptions cannot pass DLL boundaries.

Are you sure this isn't just a C++ problem? It's a well known fact that due to name mangling, C++ compiled by different compilers (even different versions of the same compiler!) can't always be called.

Re:Sorry state of dynamic libraries on Linux by saurik on Tuesday September 18, 2012 @12:52AM (#4536392)
This is due to the different libraries possibly being compiled against different versions of the C++ standard library or C runtime, and is thereby somewhat relevant to and "caused by" this specific feature (where different libraries can do this in the first place).

The same thing could happen if two people, for example, linked against different major versions of an XML parser: they would be unable to share state, as data returned from one library would have been generated with one XML library and then would be modified by another, even though the types matches for the purposes of the C compiler.

In practice, different major versions should simply be considered different libraries, and many argue it is an encapsulation mistake to allow your public interface to be bound up with details of the libraries you use; however, this kind of use case is much more likely to either work correctly or fail horribly on Linux, as opposed to ending up in some kind of weird uncanny valley of stability.

People then reference C++ with regards to this, because C++ happens to have a library that acts on tons of core data types, and yet by default (you can build setups where this doesn't happen, but it requires coordination over the DLLs you are mingling) compiles trivial parts of the algorithms inlined into your logic (so there isn't even a function boundary at all, much less code that could be shared easily).

The result is that if C++ touches your public interface, you are almost certainly, at bare minimum, wanting to share something "simple" like a string or a vector, or alternatively wanted to use exceptions somewhere. You now, however, run into the XML library issue.

Some people believe this is simply bad API design (as C++ should be treated as a library to encapsulate), while others believe it is a language deficiency. I argue that to the extent it is the latter you probably do control the DLLs that are acting as a set, and external APIs should operate under the former regime (as yes: you also have to worry about name mangling there, although compilers have gotten very good at that: we now tend to see them as part of the system ABI).

Re:Sorry state of dynamic libraries on Linux by saurik on Monday September 17, 2012 @06:23AM (#4531904)
Mach-O, the executable format used by Darwin (and hence Mac OS X) also has that feature, where it is called "two-level namespaces".
Re:Sorry state of dynamic libraries on Linux by quotemstr on Monday September 17, 2012 @06:38AM (#4531950)
Sure. Windows has one additional feature that dyld lacks, however: activation contexts[1]. You can have in the same process modAv1!foo and modAv2!foo even if modAv1 comes from c:\foo\moda.dll and modAv2 comes from c:\bar\moda.dll. (Note that the module filenames are identical.)

[1] http://msdn.microsoft.com/en-us/library/windows/desktop/aa37...

Re:Sorry state of dynamic libraries on Linux by saurik on Monday September 17, 2012 @11:34AM (#4533021)
Mach-O's two-level namespace does not work based on the one-level filename of the library on disk: it works based on the canonical full path stored in the LC_ID_DYLIB load command of the library; these names should contain the version number of the library, and so provides the functionality that you seem to be referencing (although I am very rusty at Windows and never knew how things worked at this particular level, so I'm working off of the short paragraph you just linked).

As an example:

/usr/lib/libreadline.dylib is actually /usr/lib/libedit.2.dylib

/System/Library/Frameworks/Python.framework/Python is actually /System/Library/Frameworks/Python.framework/Versions/2.6/Python

You can thereby have symbols linked from both /System/Library/Frameworks/Python.framework/Versions/2.5/Python and /System/Library/Frameworks/Python.framework/Versions/2.6/Python. "(Note that the module filenames are identical.)"

Re:Sorry state of dynamic libraries on Linux by quotemstr on Monday September 17, 2012 @02:41PM (#4534120)
Oh, cool. Thanks! My OS X is a bit rusty.