News for hackers. Stuff that matters.
Re:Static Linking Considered Harmful (2004) by danShumway on Monday September 28, 2020 @12:03PM (#24618102)
I wanted to quote a few parts of the article that I considered incorrect, but then realized I would be quoting most of the article. Short version, almost anyone who's ever tried to ship a nontrivial game on Linux should realize this article is wrong.

I particularly disagree with the idea that doing any dynamic linking in a program at all means that instantly it's just as non-portable as a fully dynamically linked program. These things are a continuum, there are degrees of fragility -- you can make a program that breaks less often when ported to new systems. See some of Ethan Lee's writing on this topic.[0]

On a meta-level, you should be instantly suspicious of any programming advice that includes quotes like:

> Conclusion: Never use [thing]!

> This has never been the case and never will be the case.

In reality, most software techniques have benefits and downsides. If a debate has survived for over a decade, that means that the answer probably isn't so completely universal that you can completely ignore one side.

I'm not going to say that there are no black-and-white correct answers in software development, but people who regularly claim to have found them should be regarded with suspicion. On average, they'll often just be oversimplifying things.

[0]: https://gist.github.com/flibitijibibo/b67910842ab95bb3decdf8...

Re:Static Linking Considered Harmful (2004) by vvanders on Monday September 28, 2020 @12:21PM (#24618307)
> I'm not going to say that there are no black-and-white correct answers in software development, but people who regularly claim to have found them should be regarded with suspicion.

I'm going to go a step further and say that the whole reason that software exists as a large profession is specifically because these trade-offs and domain and situation specific.

If we really had one-size-fits-all software as a whole would largely be a solved domain.

Re:Static Linking Considered Harmful (2004) by cycloptic on Monday September 28, 2020 @01:21PM (#24619012)
>almost anyone who's ever tried to ship a nontrivial game on Linux should realize this article is wrong.

How? I'm honestly confused what you mean here, can you explain how to build a nontrivial game on Linux while statically linking GL/Mesa/X11/Wayland/ALSA/Pulseaudio/libpng/etc, and guaranteeing that it will work on all systems and won't have any issues or security problems? Because as far as I can tell this is close to impossible. Or do you consider use of dlopen to be a different case than what the article is discussing?

Re:Static Linking Considered Harmful (2004) by danShumway on Monday September 28, 2020 @03:14PM (#24620238)
You don't need to statically link core standard libraries (you don't want to ship an entire OS), but neither should you dynamically link everything in your program.

If you go into shipping a game using only dynamic linking (as this article advocates), then you're going to run into a bunch of portability problems. I'm hard pressed to think of a reason you would ever want to dynamically link to something like vorbis or libpng for a game.

The goal is not ideological purity in either direction, it's to minimize compatibility issues. It's to look at each library and decide whether the upsides of using the system's version (assuming it has that library) override the downsides of needing to deal with all of the different systems and library versions people might be running.

Re:Static Linking Considered Harmful (2004) by cycloptic on Monday September 28, 2020 @03:44PM (#24620517)
Examples of CVEs in libpng and libvorbis are not exactly hard to find:

https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-5146

https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-7317

If your nontrivial game has an online component with voice chat or custom avatars or something like that, then you now have an attack vector. If your game is closed source, I hope you are following updates from these libraries and either updating your vendored copy or backporting patches and then immediately pushing out a new release whenever an upstream update is published. And you will likely want to be committed to doing this for however many years you plan to keep your game online.

And if you're going through all the trouble of doing that, I don't see what benefit you're getting from static linking these libraries anymore. (Unless the argument is that you're going to patch it faster than the distros)

Re:Static Linking Considered Harmful (2004) by danShumway on Monday September 28, 2020 @04:33PM (#24620949)
And if your nontrivial game doesn't have an online component, then you don't have an attack vector. For a lot of games, you're never going to deal with user-submitted audio files.

If you do have an online component or mod-loader that's affected by a vulnerability, you'll still find that the amount of work required to recompile a game and ship an update when a security release is issued for a dependency is less than the amount of work required to deal with accidental breaking changes in minor version updates, or with systems that don't install the latest package versions, or with systems that have renamed their libraries, or with systems that have stuck custom incompatible patches on top of their libraries.

If you're monitoring CVE releases for your statically linked dependencies, then when something breaks you push one patch and that's it, you're done. The alternative is that the problem never, ever goes away because people don't keep their systems up to date, and they have weird setups, and there's no patch you can ship to address that problem. You really don't want to be in a position where you're supporting multiple versions of the same library. When a security patch gets issued, you're going to do the same amount of testing regardless, so you might as well ship one version of the library at the end of that testing process that you know will work.

Aside from the support issue, there's also the fact that for security reasons you'd really like to fix most security bugs in your app yourself anyway, because, again, people don't keep their systems up to date, so it doesn't matter how quickly distros patch their libraries. If you don't ship an update that fixes the issue for everyone, then it won't be fixed for everyone. You can't trust users to update their own system libraries or not to forcibly maintain older versions, and for any app that needs to maintain a reputation of security (ie, most encrypted chat apps and password managers), you don't want your users to get hacked just because they were using a shared library ended up being vulnerable.

And even that's ignoring the fact that there are tradeoffs even for security fixes. If we're talking about a 10 year old game that's been abandoned, we'd like to have the ability to easily run it, even if we acknowledge that it might have some security holes.

But ultimately, this is really not worth fighting about. Prove Ethan wrong, try shipping dozens of commercial games with the strategy you propose and see whether or not your support requests go up or down. There's a reason game devs overall trend towards statically linking libraries. We're not doing it just to be difficult.

Re:Static Linking Considered Harmful (2004) by cycloptic on Tuesday September 29, 2020 @07:49PM (#24634415)
It seems like this is coming down to a wager that you can get security updates out faster than a distro. Short-term this is probably true, long-term I doubt it. The incentives are not there for you.

If a 10 year old game has been abandoned, I don't mind testing or recompiling it for some security updates. I've done this for lots of open source games. I personally would never work on a closed source game, so most of what you/Ethan are talking about doesn't concern me. The situation changes when you stop trying to sidestep distro maintainers, and start committing to working with them. I get that it's inconvenient to do that and you're not just being difficult, but surely you know the trade-offs.

Re:Static Linking Considered Harmful (2004) by xfer on Monday September 28, 2020 @09:48PM (#24623402)
Well, the trick is to ship your own libpng/libvorbis linked dynamically. So you only update a tiny library than half a gigabyte executable when you need a security fix. You also don't depend on the distro to provide the library.

In fact a lot of indie games that support linux do it this way, setting LD_LIBRARY_PATH with a shell script and it works well.

Re:Static Linking Considered Harmful (2004) by gridlockd on Monday September 28, 2020 @05:51PM (#24621595)
This is how it is done on Windows, which is a far more successful game platform than Linux. Obviously you need to dynamically link to OS interfaces, but you either statically link or ship with your dependencies.

On Linux, relying on system provided dependencies, especially C++ ones, is guaranteed to break over time and over different distributions. So you would have to maintain your Linux version either way, shipping with your dependencies is just less insane.

Re:Static Linking Considered Harmful (2004) by cycloptic on Tuesday September 29, 2020 @07:40PM (#24634356)
It's not guaranteed to break if you actually work with the distro maintainers. They don't want your packages breaking either.
Re:Static Linking Considered Harmful (2004) by gridlockd on Wednesday September 30, 2020 @04:28AM (#24636686)
Maintainers aren't going to keep outdated packages around forever, even if you "work with them". In effect, there is some ongoing maintenance there either way. If I just maintain my own dependencies, then I don't have to work with maintainers from a dozen distributions.
Re:Static Linking Considered Harmful (2004) by cycloptic on Friday October 02, 2020 @09:54AM (#24662597)
>Maintainers aren't going to keep outdated packages around forever, even if you "work with them".

In general, yes they will. You don't have to do much besides communicate to them that it's safe to bump a dependency and won't cause breakage. And you don't have to track down a dozen maintainers either, simply putting it in your release notes where they can find it is enough. The main reason why packages are dropped is because of lack of maintenance, possibly caused by absent or unresponsive upstream. That's what outdated means. Do you think packagers are going through all the trouble of dropping unmaintained versions of Python 2 on a whim?

Re:Static Linking Considered Harmful (2004) by klodolph on Monday September 28, 2020 @12:33PM (#24618428)
Games on Linux are a beast, aren’t they! I think doing this kind of thing successfully requires making good decisions about which dependencies you consider to be part of the system and which dependencies you want to include with your application.

For example, it is fairly reasonable to require LibSDL2 to be installed. However, if you bundle LibSDL2 with your app, you’re now directly dependent on things like X11 and ALSA.

Re:Static Linking Considered Harmful (2004) by donio on Monday September 28, 2020 @12:55PM (#24618705)
Controversial opinion but making sure that your game runs well under Wine is not the worst way to ship games for Linux.

This is especially true once the game gets older and less supported. The native Linux libraries might move on over time and the version you require might become unavailable or broken in some way but once a game works on Wine it will likely continue working forever unless you make some change to break it.

These days when it comes to my Steam library I am less worried about the Proton[1] games than the native ones and there have been several cases where I opted to run the Proton version because it worked better than native.

[1] Proton is the version of Wine Steam uses

Re:Static Linking Considered Harmful (2004) by AnIdiotOnTheNet on Monday September 28, 2020 @12:37PM (#24618484)
> I think doing this kind of thing successfully requires making good decisions about which dependencies you consider to be part of the system and which dependencies you want to include with your application.

Herein lies the problem, because Linux has absolutely no distinction between "system" and "application". Everything is part of the system.

Re:Static Linking Considered Harmful (2004) by Joker_vD on Monday September 28, 2020 @12:57PM (#24618726)
> For example, it is fairly reasonable to require LibSDL2 to be installed.

You mean, "the game's package has libsdl2 listed as a requirement"? Or "if you install the game without libsdl2 present, it will install, but will crash on launch"?

> However, if you bundle LibSDL2 with your app, you’re now directly dependent on things like X11 and ALSA.

Well, either those are present and the game will work, or you've tried to install the game on a headless server and surely you don't expect to miraculously just work?

Re:Static Linking Considered Harmful (2004) by klodolph on Monday September 28, 2020 @01:15PM (#24618951)
> You mean, "the game's package has libsdl2 listed as a requirement"? Or "if you install the game without libsdl2 present, it will install, but will crash on launch"?

Are you some kind of malevolent genie? Trying to figure out the worst possible user experience that corresponds to the words I wrote?

> Well, either those are present and the game will work, or you've tried to install the game on a headless server and surely you don't expect to miraculously just work?

The assumption here is that the only two options are either “X11+ALSA” or “headless”, and I don’t agree with that assumption. SDL2 on Linux has two video drivers and five audio drivers. Fact is, I can run horribly old Linux software that uses SDL for audio and it works fine, but old software that uses OSS or /dev/dsp is not going to work. With Wayland, I think it's not unreasonable to prefer a dependency on SDL over a direct dependency on X11.

Re:Static Linking Considered Harmful (2004) by Joker_vD on Monday September 28, 2020 @02:10PM (#24619587)
Oh, I think I understand your point now: there are old versions of SDL2 for old Linuxes, and more recent versions of SDL2 for modern Linuxes, and they use different Linux subsystems, but since they have the same API, any old software that used SDL2 still functions properly to modern Linux even though the low levels of the stack have changed dramatically.

Yes, that's a valid point actually. But ultimately it's a bet on what API will end up more durable: X11 or SDL2, isn't it?

Re:Static Linking Considered Harmful (2004) by klodolph on Monday September 28, 2020 @02:23PM (#24619754)
Yeah, it’s a bet on which API is more durable. Some other minor factors are that you might want a newer SDL anyway, because the library comes with a database of different models of gamepads (https://github.com/gabomdq/SDL_GameControllerDB) and if you link against SDL dynamically, you get support for newer gamepad models for “free”.

One of the other factors here is that the ability to play old games is in some sense about the preservation of culture. If you have an x64 Linux binary that uses SDL + OpenGL, I have a strong feeling that you’ll be able to play it 20 years from now, although it might inconvenient to get working.

Re:Static Linking Considered Harmful (2004) by m463 on Tuesday September 29, 2020 @11:14PM (#24635357)
The problem is that linux is pretty much a source-level operating system. It's designed that way philosophically and practically... or is that impractically?

Dynamic linking is probably your path if you're developing an app that is packaged and shipped with some distribution of linux and you use standard interfaces.

But yeah, a game would make this all wrong. With dynamic linking your game would last one release cycle. As soon as ubuntu 20.04 comes out, your 18.04 game would stop working. You'd get some weird shared library mismatch in your 100 dependencies and that would be that.

try this for fun:

  cd /usr/bin
  ldd mplayer
Re:Static Linking Considered Harmful (2004) by gridlockd on Monday September 28, 2020 @05:41PM (#24621514)
There are still too many people out there who think (or even insist) that you should ship your game on Linux. This has never been the case and never will be the case.