News for hackers. Stuff that matters.
Re:Broken by design: systemd by bitwize on Monday February 10, 2014 @05:54AM (#7210251)
Nice article, but at this point it's game, set, match -- systemd has won. The Linux community overwhelmingly supports it, and its advantages clearly outweigh its perceived disadvantages.

"It's not Unix" is no longer a valid critique. The old Unix design philosophy is dead. People want systems of components that function as an integrated whole. They do NOT want pieces that are generally unaware of each other (this is touted as the virtue of "loose coupling").

Yes, pid 1 does a lot. That's by necessity. It's not your grandpap's Unix anymore. Do you want to control all the system services, including the "badly behaved" ones? That functionality needs to be in pid 1. Do you want to bring the system up safely and sanely, accounting for things like hotplugged disks in /etc/fstab? That functionality needs to be in pid 1. Systemd is the correct design for modern Linux. It also brings the advantage of better integration and greater commonality across distros.

Re:Broken by design: systemd by asdfaoeu on Monday February 10, 2014 @06:25AM (#7210386)
Even on systemd that stuff isn't all handled in pid 1.

There are 6 systemd service processes running on my machine handling different components.

Snipped out the irrelevant processes.

    % systemd-cgls
    ├─user.slice
    │ └─user-1000.slice
    │   └─[email protected]
    │     ├─6612 /usr/lib/systemd/systemd --user
    │     └─6616 (sd-pam)
    └─system.slice
      ├─1 /sbin/init
      ├─systemd-udevd.service
      │ └─151 /usr/lib/systemd/systemd-udevd
      ├─systemd-logind.service
      │ └─345 /usr/lib/systemd/systemd-logind
      └─systemd-journald.service
        └─125 /usr/lib/systemd/systemd-journald
Re:Broken by design: systemd by repsilat on Monday February 10, 2014 @06:15AM (#7210337)
People want systems of components that function as an integrated whole. They do NOT want pieces that are generally unaware of each other (this is touted as the virtue of "loose coupling").

This may be true of "mainstream" users' applications, but is it really true of system software? And should we really attribute the cultural shift toward monolithic systems to shifting preferences rather than leap-first "design"?

Re:Broken by design: systemd by icebraining on Monday February 10, 2014 @06:04AM (#7210285)
Why do those things need to be on pid 1? Just asserting it is not really convincing.
Re:Broken by design: systemd by sandGorgon on Monday February 10, 2014 @06:18AM (#7210348)
Any init system needs to be pid 1 [1]

This blog post does not identify the subtle argument that upstart maintainers made against systemd at pid -1 : it also brings along with all the other services (like logind) that frameworks like Gnome depend on.

So, if Gnome wants to depend on logind and logind depends on systemd (post systemd rev-205), this means that Gnome is irrevocably coupled to systemd which must run as PID-1 [2]. This means it becomes hard for upstart to be used with Gnome.

There has been no doubt whatsover that an init system needs to run as PID-1.

[1] http://en.wikipedia.org/wiki/Process_identifier#Unix-like [2] I must add here that Gnome does include a fallback path to not depend on systemd components.

Re:Broken by design: systemd by vidarh on Monday February 10, 2014 @06:27AM (#7210390)
A part of any init system needs to be pid 1.

The question is how much of it, and whether or not Systemd piles too much stuff into pid 1 that could instead be farmed out to separate processes, handled by separate binaries, that could be more easily upgraded or replaced.

(I don't know; I've not looked at Systemd enough to have an opinion)

Re:Broken by design: systemd by sandGorgon on Monday February 10, 2014 @06:51AM (#7210467)
they are separate binaries - however all of them depend on systemd (post rev 205) and cannot be run independently.

this is because all of systemd's services rely strongly on cgroups for resource control and upstream kernel wants a single reader/writer for the cgroups API (the current filesystem model is going away). systemd is just following lockstep with what kernel wants and is becoming that API.

There has been a lot of FUD around the "PID 1" disaster, but invariably every init system will have to have a single point cgroup writer. Systemd is just ahead of the curve.

Re:Broken by design: systemd by ithkuil on Monday February 10, 2014 @12:16PM (#7212194)
http://lists.freedesktop.org/archives/systemd-devel/2013-Jun...

Anyone has more information about this from the kernel perspective ? Will systemd be the only API ? Sounds strange.

Re:Broken by design: systemd by sandGorgon on Monday February 10, 2014 @12:47PM (#7212423)
This is a reasonable evaluation of the status - https://lwn.net/Articles/575672/

Basically, if you use systemd as the init system, then yes - it is the only API [1] . If you dont want to use systemd then there are alternate implementations of the cgroup API (e.g. cgmanager - http://cgmanager.linuxcontainers.org/ ). IMHO, they are both incompatible with each other.

Lennart has a reasonable justification of why systemd is implementing it's own API [2]

[1] https://lwn.net/Articles/557111/ [2] https://lwn.net/Articles/557140/

Re:Broken by design: systemd by abofh on Monday February 10, 2014 @09:50AM (#7211318)
PID1 also comes with powers over capabilities and (in some cases) cgroups. You can do much of that with FSCaps, etc - but to control the bounding set, you have to be pid1. So - if you want init to be able to limit what processes can do, you will be pid1.
Re:Broken by design: systemd by justincormack on Monday February 10, 2014 @08:40AM (#7210940)
Linux has always supported userspace diversity, so nothing wins completely.

We are at a point where containers are going to change what people want in a Linux distro and not sure that monolithic will win.

Re:Broken by design: systemd by sitkack on Monday February 10, 2014 @11:38AM (#7211937)
Containerization will make this entire kerfuffle pointless. Running single image big machines with lots of services is a 90s and early 2000s architecture aesthetic that will fade into the background as individual services migrate to a PaaS model for enhanced modularity and mobility.

I haven't looked deeply into systemd, but from how the conversation around it is going, it would seem to be solving complexity with more complexity.

I'd like to see a mashup of CoreOS plus NixOS integrated with a cluster container management system.

Re:Broken by design: systemd by njharman on Monday February 10, 2014 @11:58AM (#7212087)
"is a 90s and early 2000s architecture aesthetic"

I hope you realize "containers" are so similar (identical compared to "single image big machines") to how mainframes worked in 70's.

Just saying nothing is new, cycles are the norm and don't be so high and superior cause your new hotness is gonna be the "old, broken shit" in a couple decades.

Re:Broken by design: systemd by sitkack on Monday February 10, 2014 @01:14PM (#7212614)
And mobile is the new desktop because there is enough computational power there. Soon, we will see containerization on mobile because of too much complexity and not enough security.

Containerization has always been a good idea. http://en.wikipedia.org/wiki/FreeBSD_jail has been around for almost 14 years. I am aware that full system virtualization dates back to the 1960s, in no way do I think that old eq bad.

Re:Broken by design: systemd by justincormack on Monday February 10, 2014 @02:49PM (#7213242)
CoreOS is of course systemd based...
Re:Broken by design: systemd by sitkack on Monday February 10, 2014 @05:53PM (#7214392)
I know, but CoreOS won't be the only lightweight OS designed for cloud deployments. I'd be pretty jazzed for a Lua or Gambit Scheme based system. It will probably be node.js though, always bet on javascript.
Re:Broken by design: systemd by justincormack on Monday February 10, 2014 @07:42PM (#7214876)
Been working on Lua based stuff that can bring up enough of a system for containers (networking and so on, still some bits missing) https://github.com/justincormack/ljsyscall
Re:Broken by design: systemd by bitwize on Tuesday February 11, 2014 @05:18AM (#7216832)
Re:Broken by design: systemd by sitkack on Wednesday February 12, 2014 @02:22AM (#7223045)
I wish l33tstart was a real thing. Maybe have it talk over nanomsg using capnproto? But the messages should be cryptographically signed. Would it handle logging as well? Could those log messages get processed through summingbird or storm? Ok, Clojurescript -> Gambit Scheme -> Native exe
Re:Broken by design: systemd by d0 on Monday February 10, 2014 @06:06AM (#7210293)
daemontools doesn't run as pid 1 and achieves all of this.

http://cr.yp.to/daemontools.html

Re:Broken by design: systemd by FooBarWidget on Monday February 10, 2014 @06:11AM (#7210315)
Re:Broken by design: systemd by Fasebook on Monday February 10, 2014 @07:14AM (#7210554)
That comment is insane. Why would log files need journaling? They should be serialized.

How is adding a line to a run script "duplicating logic badly"?

Coupling is clearly bad, as evidenced by ... 30+ years of software design, but is a necessary evil. I don't see any reason why it's necessary in such a low level system component. "this is not rocket science" you don't impose dependencies on something you want to be reliable.

Daemontools is clearly the most mature solution, from a perspective of stability. It's probably why nobody wants to use it, there's nothing to play with or exploit for profit because it already works.

Re:Broken by design: systemd by bitwize on Monday February 10, 2014 @08:48AM (#7210989)
That comment is insane. Why would log files need journaling? They should be serialized.

systemd-journald provides tampering-attestation features that plain-text logs do not, and cannot, provide. It's fundamentally more secure.

Re:Broken by design: systemd by peterwwillis on Monday February 10, 2014 @09:54AM (#7211344)
Fallacy: having owned the user that writes logs, I cannot fuck with your logs if journaled.

Reality: if I owned the user writing logs, I can do anything I want with the logs, no matter how they're stored.

Re:Broken by design: systemd by hdevalence on Monday February 10, 2014 @10:39AM (#7211567)
No, you cannot do anything you want with the logs.

Here is something you cannot do: you cannot alter the logs without anyone noticing, since log entries have a rolling hash.

This prevents an attacker from being able to modify logs without detection, and was implemented in systemd after the postmortem on the kernel.org breakin, i.e., it was developed to counter an actual threat.

I would suggest you read up on what log attestation actually does.

Re:Broken by design: systemd by peterwwillis on Monday February 10, 2014 @11:48AM (#7212011)
Logs can be modified in real time by being intercepted during writes to the buffer before the new hash is applied. Because of this, only the logs up to the intrusion can be preserved, and even then if the old message/hash/key/etc is still in memory, the old messages can be modified.

The only truly secure way to record messages up to the point of break-in is a one-way remote log. Any new messages after break-in is subject to falsification.

Re:Broken by design: systemd by bct on Monday February 10, 2014 @11:59AM (#7212089)
> The only truly secure way to record messages up to the point of break-in is a one-way remote log.

And if you have this then you don't need a fancy cryptographic syslogd to detect tampering.

Re:Broken by design: systemd by peterwwillis on Monday February 10, 2014 @02:09PM (#7212967)
The fancy cryptologic syslogd is fallible as I mentioned previously, so why depend on it? And really if you really want to cover your tracks you can just cause random disk corruption over the whole disk (and just happen to damage the journal in the process). Things will start failing rapidly, fsck will later show itself fixing the disk corruption, and it will be assumed to be a hardware error and the incident ignored.

Security half-measures do not mean you are secure. If it can be hacked, it will be hacked, and then what was the point of sacrificing your whole system's init system?

(Also you could just replace syslogd with a journaled syslogd, rather than replacing your entire init system... but I guess that's off-topic?)

Re:Broken by design: systemd by bct on Monday February 10, 2014 @04:10PM (#7213853)
I'm agreeing with you.
Re:Broken by design: systemd by Fasebook on Monday February 10, 2014 @02:39PM (#7213173)
>rolling hash

FAIL. You just wrote outdated software. Under this dumbass idea of journaling, which by defition is there to allow editing.., the best thing you can do is built a rolling function, which basically means coping the data, which basically means increasing your attack surface.

Re:Broken by design: systemd by andor on Monday February 10, 2014 @11:02AM (#7211730)
The journal sealing provides forward security. The sealing keys are exchanged regularly, and the old keys are deleted safely. A verification key, which is stored on another device or offline, can be used to calculate the sealing key for any given moment. While an attacker can change the logs, this will always be visible in an audit.

http://lwn.net/Articles/512895/

https://eprint.iacr.org/2013/397.pdf

Re:Broken by design: systemd by d0 on Monday February 10, 2014 @11:08AM (#7211762)
the old keys are deleted safely

Assuming nothing has managed to redirect those unlinks for example...

Re:Broken by design: systemd by andor on Monday February 10, 2014 @03:25PM (#7213543)
I guess you're right. The mechanism is forward-secure, which means for an attacker that tampering the logs is the first thing to do, not the last ;-)

As soon as the attacker manages to extract the state of the key generator from memory at one point of time, the log entries from that and the following sealing periods can be modified and cleanly sealed.

Re:Broken by design: systemd by asabjorn on Monday February 10, 2014 @10:53AM (#7211656)
From what I understand systemd can optionally store a copy of the logs on a remote server. The key to modify the logs on the remote server is changed in such a way that even if a system is compromised the log copy on the remote system can not be changed.

These remote copies of the logs could actually be used to detect log-tampering and 0day exploits.

Re:Broken by design: systemd by Fasebook on Monday February 10, 2014 @03:27PM (#7213566)
>From what I understand systemd can optionally store a copy of the logs on a remote server.

anything can have any feature, what matters is how it's built.

Re:Broken by design: systemd by edwintorok on Monday February 10, 2014 @10:53AM (#7211663)
rsyslog allows you to sign your logs if you want to: http://www.rsyslog.com/how-to-sign-log-messages-through-sign...
Re:Broken by design: systemd by aidenn0 on Monday February 10, 2014 @12:34PM (#7212304)
daemontools can't reliably monitor daemons that reparent themselves to init or kill their pgroup.

I have written hacks to work around this (I have a gentoo VM that uses supervise as an openRC replacement, and have several hacks to work around this issue). It's a pain in the ass, and one reason why systemd runs in pid 1.

I prefer daemontools, but it's not perfect; it made a different set of tradeoffs.

Re:Broken by design: systemd by gnoway on Monday February 10, 2014 @08:43AM (#7210957)
How does svscan handle service dependencies?
Re:Broken by design: systemd by d0 on Monday February 10, 2014 @09:06AM (#7211094)
It doesn't. That's the point. Nothing in daemontools does. Dependencies are bad.
Re:Broken by design: systemd by deong on Monday February 10, 2014 @09:43AM (#7211281)
I'm not sure if you're making a point more subtle than I'm reading, but when most people in the systemd debate say "dependencies are bad", they mean that having loads of things depend on the init system is bad.

The comment you're responding to is talking about dependencies in boot services, e.g., sshd depends on having the network interface up. Systemd does a great job of handling this sort of thing, allowing services to start in parallel, but ensuring that things only start after all the pieces they rely on have started. That sort of dependency isn't "bad" -- it just is. You can't remove a dependency like this, it reflects reality.

That said, I really don't care for systemd. I think the complexity it adds more than counteracts the benefits it provides, but I'm old and didn't mind SysV or BSD init.

Re:Broken by design: systemd by d0 on Monday February 10, 2014 @09:59AM (#7211371)
I'm saying that the dependency on the network being up is bad for services. An example of a bad case: if the network interface is hotplugged or down and you're running Redis, do you want it to restart and have to drag the entire AOF file off disk just because you yanked the interface card because it was failed or failing? Your 20 second hotplug problem becomes a 15 minute block of IO coming off disk because you had to restart the process.

So many problems come from such a simple dependency.

Designing all services to handle and expect failures and just let init restart regardless of the state is the RIGHT solution.

Re:Broken by design: systemd by ChuckMcM on Monday February 10, 2014 @12:13PM (#7212173)
Linux was never UNIX. Influenced yes, but not UNIX. That said, in the FOSS world it is never 'game, set, match' it is 'game, set, fork' :-). For what its worth though, these sorts of debates have happened forever, MOTIF vs CDE, Streams vs Sockets, Signals vs Events, TCP vs OSI, Etc.
Re:Broken by design: systemd by stormbrew on Monday February 10, 2014 @06:04AM (#7210286)
I'm not so sure this is entirely true, especially with pid namespaces becoming a real thing. The true pid1 could be greatly simplified for the sake of system stability while providing sub-pid1s that handle actual service, device, and application management.
Re:Broken by design: systemd by adobriyan on Monday February 10, 2014 @06:28AM (#7210393)
Everyhing in pidns is destroyed when pidns init dies.
Re:Broken by design: systemd by stormbrew on Monday February 10, 2014 @06:57AM (#7210489)
I'm not sure what you're getting at here. Yes, that is obviously true. As is the case with the true pid1 init. Care to expand?
Re:Broken by design: systemd by adobriyan on Monday February 10, 2014 @07:02AM (#7210509)
I mean if you protecting yourself from panic by moving stuff from real PID 1 into pidns root, you're doing it wrong. pidns root will segfault and kernel will SIGKILL everything inside which is equivalent to good old panic.
Re:Broken by design: systemd by stefantalpalaru on Monday February 10, 2014 @06:22AM (#7210367)
> systemd has won

Not in the Gentoo ecosystem where we have OpenRC: https://en.wikipedia.org/wiki/OpenRC

Re:Broken by design: systemd by zx2c4 on Monday February 10, 2014 @09:30AM (#7211211)
We'll likely switch to systemd, over at Gentoo, when we need to. As we discussed at FOSDEM, it's a matter of 'when' rather than 'if'.
Re:Broken by design: systemd by theBlackDragon on Tuesday February 11, 2014 @05:41AM (#7216898)
If that switch happens and systemd is forced on us I'll be dumping Gentoo after 12 years.

systemd gives me nothing that openrc doesn't already provide and it messes with a lot of things that I do care about (eg. logging, cron).

If RedHat is so intent on forcing their junk through everyone's throat it's time to move back to an OS where technical insight reigns instead of corporate money and marketingspeak.

Re:Broken by design: systemd by makomk on Tuesday February 11, 2014 @05:51AM (#7216929)
To be honest, if Gentoo moves to systemd it'll probably be the final straw that pushes me to switch back to Windows after a decade or so of Linux use. Things have just being getting more and more buggy and unreliable; I'm already on the verge of switching just so I don't have to deal with the state of audio on Linux these days. Every PulseAudio version introduces new and fun breakages and they don't do stable releases anymore, so at best it'll get fixed in the next release which will have a bunch more poorly-tested code that breaks something else.
Re:Broken by design: systemd by stefantalpalaru on Tuesday February 11, 2014 @06:14AM (#7217016)
I have been able to keep PulseAudio out of all my Gentoo systems so far. I seriously doubt anybody can impose systemd on me.
Re:Broken by design: systemd by nailer on Monday February 10, 2014 @06:58AM (#7210497)
Gentoo is still popular as it's the basis for ChromeOS and CoreOS, but CoreOS uses systemd and I'm not sure re: ChromeOS.
Re:Broken by design: systemd by saljam on Monday February 10, 2014 @07:53AM (#7210697)
I was under the impression ChromeOS used upstart.
Re:Broken by design: systemd by mdeslaur on Monday February 10, 2014 @09:03AM (#7211078)
ChromeOS uses upstart.
Re:Broken by design: systemd by edwintorok on Monday February 10, 2014 @07:21AM (#7210575)
Also Debian is still on sysvinit, and there is a vote being held to decide what init system to use for jessie as default.
Re:Broken by design: systemd by akerl_ on Monday February 10, 2014 @07:36AM (#7210627)
That vote has almost total agreement that systemd will be the new init system. At this point, they're stuck in a quagmire of other bureaucratic nonsense and debates over related issues like whether packages will be allowed to require a particular init system.
Re:Broken by design: systemd by zimbatm on Monday February 10, 2014 @08:08AM (#7210755)
Or maybe they need to figure out how to keep the non-Linux ports working since systemd is highly tied to that particular kernel.
Re:Broken by design: systemd by ibotty on Monday February 10, 2014 @01:11PM (#7212592)
neither the hurd nor the bsd port use the debian linux sysinit currently.
Re:Broken by design: systemd by Fasebook on Monday February 10, 2014 @05:04PM (#7214187)
If this were true, why would you bother saying it.
Re:Broken by design: systemd by bitwize on Friday February 14, 2014 @01:06PM (#7239952)
Because the holdouts in the init wars debate are whiners who are trying to obstruct useful progress being made on the Linux platform, e.g., Ian Jackson.