News for hackers. Stuff that matters.
Re:Broken by design: systemd by oofabz on Monday February 10, 2014 @06:09AM (#7210305)
> PID 1 brings down the whole system when it crashes. This matters because systemd is complex.

The Linux kernel is much more complex and also brings down the whole system when it crashes.

> Attack Surface

Sysvinit runs shell scripts and has known race conditions. A system like that is not secure by design. It is as secure as it is because exploits were fixed one by one over the years.

> Reboot to Upgrade

You must also reboot to upgrade sysvinit.

Re:Broken by design: systemd by lelf on Monday February 10, 2014 @07:12AM (#7210543)
> The Linux kernel is much more complex and also brings down the whole system when it crashes

Yeah. And there is nothing good about it.

> Sysvinit runs shell scripts and has known race conditions

Bugs can be everywhere. But the smaller the paramount part the better.

> You must also reboot to upgrade sysvinit

You don't

Re:Broken by design: systemd by sspiff on Monday February 10, 2014 @07:40AM (#7210644)
>> The Linux kernel is much more complex and also brings down the whole system when it crashes

> Yeah. And there is nothing good about it.

I beg to differ, and I think a lot of people will disagree with you. Regardless, your snarky remark fails to state any problems with the Linux kernel, or identify what you consider a "good" kernel.

Re:Broken by design: systemd by _quasimodo on Monday February 10, 2014 @08:18AM (#7210797)
I think he meant there is nothing good about complexity and bringing down the whole system on a crash. For the Linux kernel this is probably an unfortunate necessity, for an init system it is not.
Re:Broken by design: systemd by ptr on Monday February 10, 2014 @08:09AM (#7210763)
He's probably thinking about some kind of micro kernel (which are pretty cool), or redundant CPUs (which probably wouldn't help much)?
Re:Broken by design: systemd by uggedal on Monday February 10, 2014 @06:21AM (#7210365)
> You must also reboot to upgrade sysvinit.

The whole point of the article is that you don't need to upgrade sysvinit or another minimal /sbin/init like those found in the BSDs.

Re:Broken by design: systemd by adobriyan on Monday February 10, 2014 @07:11AM (#7210539)
The argument is kinda weak given how many kernel updates are there and how often they come (and kernel upgrade _requires_ reboot regardless of PID 1).
Re:Broken by design: systemd by michaelmior on Monday February 10, 2014 @07:36AM (#7210626)
Not necessarily. Ksplice[1] for example allows the kernel to be upgraded without rebooting.

[1] https://www.ksplice.com/

Re:Broken by design: systemd by epsylon on Monday February 10, 2014 @07:59AM (#7210720)
You have completely missed the point of the article.

So if not systemd, what? Debian's discussion of whether to adopt systemd or not basically devolved into a false dichotomy between systemd and upstart. And except among grumpy old luddites, keeping legacy sysvinit is not an attractive option. So despite all its flaws, is systemd still the best option?

No.

After which he proceeds to explain how to solve points 1, 2, and 3 by moving most tasks out of PID 1.

Re:Broken by design: systemd by dredmorbius on Monday February 10, 2014 @06:57AM (#7210493)
It is as secure as it is because exploits were fixed one by one over the years.

That's largely how things get secure. Though pre-emptive security through code audits and library rewriting (the OpenBSD approach) also helps.

Init scripts have had those years. Systemd has not.

Re:Broken by design: systemd by akerl_ on Monday February 10, 2014 @07:34AM (#7210614)
Yes, but we didn't decide to not use sysvinit on those grounds.

It's quite a nifty case to make, that we can't use a new init system because we haven't used the new init system for a long time already to find the bugs.

Re:Broken by design: systemd by dredmorbius on Monday February 10, 2014 @09:39AM (#7211257)
It's quite a nifty case to make, that we can't use a new init system

That's not the case I was making. Rather it was oofabz's apparent criticism of sysvinit's security profile based on the fact that bugs were found and removed individually. That's not a particularly valid criticism.

There are more valid ways of establishing software quality based on bug estimation (defects found, estimated undiscovered defects remaining). The rate with which init scripts are changed (the fact that they can be modified, possibly erroneously, by individual systems admins is another consideration). A system that's been specifically designed to be resistant to coding and use errors (again: OpenBSD) is better. Some init script systems are better than others, and I've found the Debian scripts to be simpler, cleaner, and more robust than Red Hat's (which use multiple levels of includes and indirection).

In the case of systemd, I've found the touted benefits ("faster booting" -- ORLY? I prefer not having to boot/reboot systems), complexity, novelty, and track-record of the primary developer to be quite troubling. Technology is a complexity management system. Much of the problem in the GNOME project comes from failing to manage complexity appropriately (oversimplifying a complex space, creating complex tools with more levels of indirection (gconf and related utilities). It's a train wreck I saw going off the rails over a decade ago. I've got similar fears for systemd.

Yes, booting is more complex, some systems (especially cloud deployments) may benefit from expedited startups, dynamic management of devices and services can be a benefit. But off of that in PID 1? And don't even get me started on the logging aspects.

I'll be sitting this one out as long as I can.

Re:Broken by design: systemd by DrJ on Monday February 10, 2014 @12:53PM (#7212474)
> (defects found, estimated undiscovered defects remaining)

how does one estimate the latter? (Actually curious if this is a known thing)

Re:Broken by design: systemd by dredmorbius on Tuesday February 11, 2014 @12:57AM (#7216046)
It's a known thing. Look up "software defect estimation".

The methods aren't too dissimilar to how population ecologists estimate the population of a wild animal population (you tag individuals and note the rate at which you're repeated re-tagging the same ones), the estimated total falls out via statistics.

It's not a method that's applied frequently even within organizations (I've worked in and with QA numerous times), and my copy of Cem Kaner's Testing Computer Software doesn't seem to address the matter. Boehm's Software Engineering Economics discusses software reliability modelling at page 181 (Chapter 10: Performance Models and Cost-Effectiveness Models).

IEEE has "An efficient defect estimation method for software defect curves" which looks like it should cover the area, membership or purchase required:

http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=124539...

"Applying Software Defect Estimations: Using a Risk Matrix for Tuning Test Effort", James Cusick, Wolters Kluwer

http://arxiv.org/pdf/0711.1669.pdf

BUG COUNT ESTIMATION: http://www.testdesigners.com/bugnets/bugcountestimation.html

References Capers-Jones, leading authority in the field.

Many methods start with an assumption that software coding is essentially a progress of inserting bugs into code at a known rate (and there are studies which have established such rates with fairly high levels of confidence). So debugging and QA are then a bug removal function. Knowing what your bug insertion rate was allows you to estimate how many of those bugs you've removed.

That's the theory.

Re:Broken by design: systemd by DrJ on Tuesday February 11, 2014 @02:13AM (#7216302)
this just makes me want to push spec review, coverage/unit testing, and smoke testings a more stringent requirement for new codes :\
Re:Broken by design: systemd by ZeroGravitas on Monday February 10, 2014 @08:23AM (#7210826)
That actually strikes me as a more reasonable objection than many I've heard. On the other hand, systemd is actually in use in various places, so bugs are being looked for and found, so this process doesn't begin with Debian's adoption and you're back to balancing probababilities the same as you would with any change.
Re:Broken by design: systemd by belorn on Monday February 10, 2014 @08:46AM (#7210979)
> Debian's adoption

In this case, adoption means moving systemd from being optional to default. Debian has had systemd in use for quite a while now with current statistics at 7.5% of all installation (accordingly to popcon http://qa.debian.org/popcon.php?package=systemd).

Re:Broken by design: systemd by bbanyc on Monday February 10, 2014 @09:45AM (#7211289)
Package names are a bit confusing here. The systemd package includes all the binaries - one of which, logind, is required by GNOME - but doesn't install systemd as /sbin/init and doesn't boot with it by default. The package to do that is called systemd-sysv, for reasons having to do with the special treatment of sysvinit as a "required" package in dpkg.

So 7.5% of Debian systems have it installed, but only about 0.3% (plus however many of that 7.5% that are setting init=systemd in GRUB) are using it.

Re:Broken by design: systemd by ibotty on Monday February 10, 2014 @01:07PM (#7212558)
setting init=/sbin/systemd in the kernel cmdline is the recommended way, so i guess the 7.5% do count as using systemd.
Re:Broken by design: systemd by pipy on Monday February 10, 2014 @08:40AM (#7210936)
Systemd is controlled by Red Hat in a way in which critical system components including kernel haven't been controlled before. Not by single corporate entity.

That's what we know about this company from an old (2007) article:

> “When we rolled into Baghdad, we did it using open

> source,” General Justice continued. “It may come as a

> surprise to many of you, but the U.S. Army is “the” single

> largest install base for Red Hat Linux. I'm their largest customer.” [1]

It is better to go with a grass-roots solution, even the one technically inferior, that isn't being influenced by one single vendor or government.

[1] http://archive09.linux.com/feed/61302

Re:Broken by design: systemd by mateuszf on Monday February 10, 2014 @11:14AM (#7211799)
> It is better to go with a grass-roots solution, even the one technically inferior, that isn't being influenced by one single vendor or government.

If politics was a problem the most practical way would be to just fork systemd because it has friendly license and is feature rich.

Re:Broken by design: systemd by baldfat on Monday February 10, 2014 @01:10PM (#7212589)
So we don't trust Red Hat because it's customer is the US Army?

I also would say that every critical system components are control by different ententes. BUT don't like systemD you can use any of a dozen options.

Re:Broken by design: systemd by dredmorbius on Tuesday February 11, 2014 @12:58AM (#7216049)
I'd be more concerned about RH and its own incentives (look at the clusterfuck that's GNOME) than the US Army specifically.
Re:Broken by design: systemd by pipy on Tuesday February 11, 2014 @05:54AM (#7216940)
Sure. I have a gut feeling that GNOME reworking was done solely to make trouble for Canonical.

The Interface Stability Promise [1] by systemd team is just a promise, nothing more. I wonder if Red Hat will keep it if it decides that it no longer serves their bottom line.

[1] http://www.freedesktop.org/wiki/Software/systemd/InterfaceSt...

Re:Broken by design: systemd by Elhana on Monday February 10, 2014 @10:15AM (#7211451)
Linux kernel is maintained by sane people, unlike systemd/udev: http://article.gmane.org/gmane.linux.kernel/1369384
Re:Broken by design: systemd by tonyskn on Monday February 10, 2014 @07:14AM (#7210553)
> Reboot to Upgrade

This is not needed anymore. Check systemd's manual for 'systemd daemon-reexec'

Re:Broken by design: systemd by pmr_ on Monday February 10, 2014 @08:07AM (#7210752)
The article specifically addresses this feature, but I'm not knowledgeable enough about the topic to judge the quality of the argument.
Re:Broken by design: systemd by makomk on Monday February 10, 2014 @08:56AM (#7211027)
So, I used to vaguely help develop another proposed init replacement a few years ago, and it's entirely correct. The only way to upgrade init is to have it dump state and re-exec() itself, and there are theoretical failure modes that will cause init to crash before it's even possible to set up a crash handler to catch the problem, causing a kernel panic.

What's more, supporting upgrades this way means you need to support every previous state file format, and the more state and features there are the harder this is to do. (Seriously, it was a pain in the butt to keep track of.) systemd is developed primarily by and for Red Hat who avoid this problem by not supporting upgrades to a new distro release from within the running system - but most other major distros do and will likely be bitten hard by the issue.

Without support for this you can't even reboot cleanly after upgrading init or any of the libraries it uses, because init will keep old deleted executable open and make it impossible to remount root as read-only (an obscure quirk of Unix filesystems that basically only init developers need to know). You don't just need to reboot to upgrade init, you need to boot into a special upgrade environment that doesn't use the system's init, and Fedora does. That's why Fedora needs daemon-reexec - otherwise they wouldn't even be able to apply security updates to glibc without rebooting first. Compare this with kernel upgrades, which can be installed immediately and simply don't take effect until the next reboot.

Re:Broken by design: systemd by andor on Monday February 10, 2014 @10:15AM (#7211452)
Without support for this you can't even reboot cleanly after upgrading init or any of the libraries it uses, because init will keep old deleted executable open and make it impossible to remount root as read-only

Lennart Poettering on how this is done in systemd:

https://plus.google.com/+LennartPoetteringTheOneAndOnly/post...

Re:Broken by design: systemd by makomk on Tuesday February 11, 2014 @05:45AM (#7216907)
That's quite a clever solution - except that upon reading what Lennart says in the comments, systemd-shutdown is apparently closely coupled with systemd and receives enough state from it that it cannot be used with any other init system. So I'm not sure it's safe to rely on that for any major upgrade. (Not entirely sure why it needs all that state either.)
Re:Broken by design: systemd by InclinedPlane on Monday February 10, 2014 @06:14AM (#7210335)
See what happens when you kill PID 1 on a system using sysvinit.
Re:Broken by design: systemd by munchhausen on Monday February 10, 2014 @07:03AM (#7210514)
On Linux, nothing:

The only signals that can be sent to process ID 1, the init process, are those for which init has explicitly installed signal handlers. This is done to assure the system is not brought down accidentally. (From kill(2))

In practice, you will find that:

    kill(1, SIGKILL)                        = 0
results in exactly...no effect.
Re:Broken by design: systemd by nailer on Monday February 10, 2014 @07:12AM (#7210545)
It's also because Unix doesn't generally believe in being forgiving (or, UX in general):

    someapp &

    [job 1 backgrounded]

    kill %1

    (job 1 is killed - yep, you can use kill for jobs)
Miss a percent there? You've just sent a TERM to init.
Re:Broken by design: systemd by srd on Monday February 10, 2014 @09:10AM (#7211123)
Which won't matter if you're a normal user. If you're root - why are you using the job system to kill your jobs?
Re:Broken by design: systemd by nailer on Monday February 10, 2014 @09:13AM (#7211139)
> If you're root - why are you using the job system to kill your jobs?

Probably because you know the features of your shell and enjoy productivity.

Re:Broken by design: systemd by yk on Monday February 10, 2014 @11:54AM (#7212060)
Because thats a job for kill.
Re:Broken by design: systemd by belorn on Monday February 10, 2014 @08:53AM (#7211008)
I wonder what would happen if you used OOM_Killer on process ID 1.
Re:Broken by design: systemd by makomk on Monday February 10, 2014 @09:10AM (#7211124)
PID 1 is meant to be immune to the OOM killer. Last time I did development work in that area, it was also impossible to attach a debugger to PID 1 without patching the kernel, and although it worked I'm not sure how safe it was to do so.
Re:Broken by design: systemd by zx2c4 on Monday February 10, 2014 @09:33AM (#7211225)

    zx2c4@thinkpad ~ $ sudo strace -p 1
    Process 1 attached
    select(11, [10], NULL, NULL, {1, 122329}) = 0 (Timeout)
    stat("/dev/initctl", {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0
    fstat(10, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0
    stat("/dev/initctl", {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0
    select(11, [10], NULL, NULL, {5, 0})    = 0 (Timeout)
    stat("/dev/initctl", {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0
    fstat(10, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0
    stat("/dev/initctl", {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0
    select(11, [10], NULL, NULL, {5, 0}^CProcess 1 detached
    <detached ...>
Ptracing seems to work for me.