News for hackers. Stuff that matters.
Re:What should we expect from a modern init system? by brokenparser on Monday January 20, 2014 @12:15PM (#7090549)
I expect an init system to be an init system, not to also be pm-utils, (x)inetd, acpid, syslogd, watchdog, cron and atd. I also expect it to have configuration files I can change, not to have everything symlinked to /usr. (If I can even find the right configuration file in that mess of directories, that is.)
Re:What should we expect from a modern init system? by exDM69 on Monday January 20, 2014 @01:41PM (#7091072)
> I expect an init system to be an init system, not to also be pm-utils, (x)inetd, acpid, syslogd, watchdog, cron and atd.

The problem with this traditional unix approach is that in addition to init, pm-utils, inetd, cron, etc you end up having thousands of lines of shell scripts that glue all this together in a big brittle mess.

While this approach may be fine for servers which boot once and stay on for ages, are connected to fast wired network and are generally rather stable configurations, I don't want such a system for my desktop, laptop or mobile device.

I expect modern devices to be able to deal with changes in power supply (battery/mains/low battery), connecting and disconnecting devices (storage, peripherals, audio devices, etc) and changes in network connectivity (intentional disconnection as well as network failure). This may involve starting and stopping services, mounting and unmounting partitions and taking other actions when the physical configuration changes.

I also expect my computers to boot and shutdown quickly and effectively.

Some of this is already doable in other systems, like the traditional Debian network setup (originally built in the era of wired networks) which can run scripts on connect/disconnect. But the situation ends up being what I described above: thousands of lines of brittle shell scripts.

I welcome the changes and the effort put in systemd and other init systems. Traditional unix init just doesn't cut it any more, in particular on personal desktop and mobile devices.

Re:What should we expect from a modern init system? by Zancarius on Monday January 20, 2014 @02:16PM (#7091245)
I had my reservations when Arch shifted to systemd. It seems like a tremendous break from Unix philosophy and a blatant violation of everything I knew.

I'm glad they did, though. Unit files are far easier to write and having a supervisor behavior in the init system alleviates a tremendous amount of administrative overhead. That's not to say shell scripts are difficult, but they can break in mysterious ways. Though I imagine the same is true for systemd, from an end user perspective, its implementation (once you understand it) requires less effort.

There are those who disagree, and I respect that. But the improvements to my desktop boot time are substantial. And writing a quick unit file to run something on boot requires little mental overhead. Just a glance over the manpage is enough to get started. sysvinit scripts on the other hand... well, maybe my memory is really poor, but if it's been more than a few months since I've written one for the specific system I'm targeting, it takes substantially longer. ;)

So, it's likely I'm stupid or systemd is easier. Or both. Probably both.

Re:What should we expect from a modern init system? by unhammer on Tuesday January 21, 2014 @05:06AM (#7094348)
I had the same experience, would not have expected to be so pleasantly surprised by an init system :) And writing unit files for systemd was fantastically easy.

systemd actually looks like it was designed based on experience. "Worse is Better" as a development philosophy works for getting stuff up and running and finding out what the real problems are so that you don't make solutions for non-existent problems. But when enough time has passed, and enough glue and pieces of string accumulated, it's time to take a step back and consider a redesign.

Re:What should we expect from a modern init system? by Zancarius on Tuesday January 21, 2014 @01:17PM (#7097103)
I think that's my biggest problem with shell-scripts-as-an-init-system. There's nothing inherently wrong with it, but sooner or later, the only way to correct for broken or missing functionality is (to borrow your term) to add more glue. And if you want more correct behavior, you either have to wait for the distro maintainers to fix it or fix it yourself only to merge your changes back in on the next OS update.

That's another thing I like about systemd. With the separation of "stock" unit files in /usr/lib and user-supplied ones in /etc/systemd, you can override the ones distributed by your OS vendor without worrying about updates breaking your changes. That's already saved me once or twice during Arch's transition from sysvinit to systemd, in fact.

One thing that comes to mind is an issue I had some time back with running a proprietary service in a manner that would allow the init system to control it when it forked itself a number of times and spun up separate processes. start-stop-daemon was confused by this behavior and it required some tricks to collect the approriate PID just to be able to stop the service (I may be wrong, but I think it was the TeamSpeak 3 daemon which behaves weirdly). systemd's use of cgroups ignores this rubbish entirely, because it's effectively impossible for systemd to lose track of what processes belong to which service. Want to kill the service? No problem.

Some in this thread seem convinced that systemd is a solution looking for a problem, but I suspect some of them lack experience with systemd. Or stubbornly cling to their ideals of what an init system should do. I like the unification of a supervisor-type service and the init system. It makes my job easier. And that's not even half of what systemd is capable of!

Re:What should we expect from a modern init system? by midas007 on Tuesday January 21, 2014 @07:19AM (#7094785)
This is not the fault of unix, it's well suited for static systems. It's the fault of implementers trying to use something that was designed for an entirely different paradigm.

Future apps will need to copy themselves and their data to other devices, perhaps even suspend on one device and wake-up on another.

Making changes for one use case at the expense of another without knowing the history is an ignorant change for churn's sake. Churn is not productivity, but the appearance of doing something.

Re:What should we expect from a modern init system? by Sssnake on Monday January 20, 2014 @06:01PM (#7092278)
>The problem with this traditional unix approach is that in addition to init, pm-utils, inetd, cron, etc you end up having thousands of lines of shell scripts that glue all this together in a big brittle mess.

No, I don't. Typical linux distros may, but I sure as hell don't. I have a very simple, easy to read and debug set of shell scripts that work very well: http://www.openbsd.org/cgi-bin/cvsweb/~checkout~/src/etc/rc?...

Re:What should we expect from a modern init system? by hkhanna on Monday January 20, 2014 @01:08PM (#7090882)
The parent comment captures why systemd makes me uneasy, and why I moved from Arch to Debian when I noticed Arch had replaced init with systemd.

The Art of Unix Programming (the book) suggests that software should do one thing and do it well, with easy, plain-text interfaces to its input and output. Systemd tries to do everything -- replacing by default so many core system components like syslog and cron.

But what really bothered me was when I realized that systemd stores all the system logfiles in binary by default! I couldn't believe it. All the system log files: boot, kern, messages, whatever, were stored in some binary format. At that point, rather than research how this new binary log file was implemented, I abandoned Arch and systemd for the init system that had served me well for 15 years.

Log files don't need to be in binary. It means we can't use our standard tools: cut, sort, grep, awk, etc. It means we can't use scripts we have written built on those tools we may have built over decades of working with UNIX.

I guess my point is, systemd, by storing things in binary format and by trying to be the 'everything' daemon, violates two core UNIX programming philosophies: use plain-text interfaces & do one thing well.

Re:What should we expect from a modern init system? by danudey on Monday January 20, 2014 @01:39PM (#7091057)
Log files being in binary lets you do things like filter based on specific fields rather than on more complicated string matching. For example, you can get log file messages from postfix's smtp process (as opposed to its smtpd process) with one command (journalctl /usr/lib/postfix/smtp), which you can then cut/sort/grep/awk/etc.

You can also follow the logs with journalctl -f and go from there as well.

The problem that this solves is the classic 'what log file does this daemon log this error level to on this system', which varies a surprising amount between daemons, versions, distributions, etc.

I don't actually care which log file I look at, I just want to know what errors monit is getting. systemd and journalctl provide that handily.

I would agree with an approach to have systemd send log events to e.g. journald (or syslog-ng, or whatever the user wanted) to separate that functionality out, however.

Re:What should we expect from a modern init system? by Zancarius on Monday January 20, 2014 @02:10PM (#7091216)
I don't mind journalctl in spite of my initial reservations, and as you point out with -f, it's not all that different from tailing a text log. I do suspect binary logging isn't the only reason hkhanna dumped Arch for Debian. It smells of nitpickery to me.

Using journalctl instead of hunting for the appropriate log files in /var/log seems to offer some degree of platform unity. Maybe it's not the right answer, but it works well enough. My primary complaint is how long it can take on a noisy system to filter a large log, but it's not exceptionally noisome. And most of the time, I just want to follow the log.

> I would agree with an approach to have systemd send log events to e.g. journald (or syslog-ng, or whatever the user wanted) to separate that functionality out, however.

While it probably isn't what you mean as you attach the caveat of task separation, it is possible for syslog-ng to collected logging from systemd's journal. It's a little flaky but it works. :)

Re:What should we expect from a modern init system? by hkhanna on Monday January 20, 2014 @03:31PM (#7091580)
>I do suspect binary logging isn't the only reason hkhanna dumped Arch for Debian. It smells of nitpickery to me

You are right, of course. Binary logging was not the only reason I ultimately switched. The big reason was that I needed a distribution that did not change so frequently (although I love what Arch is doing). Stability was so important for my use case, though, that I ended up switching to Debian Stable.

Although journalctl does work well, I admit, I wonder if the advantages could have been accomplished without the binary file format. So many people have homegrown log parsing tools that need to be re-written to handle a binary file format. Or at least re-written to call journalctl as an intermediary.

Re:What should we expect from a modern init system? by Zancarius on Monday January 20, 2014 @04:45PM (#7091951)
> The big reason was that I needed a distribution that did not change so frequently

Yeah, this is a perfectly valid reason and also why I tend not to use Arch on servers (as much as I would like to otherwise, though...). Both have their use cases, and I love Arch for desktops, but it's not a particularly great match for situations where rapid, near-constant change is unwarranted, unnecessary, or problematic. Arch is also a distro that requires a fair amount of love and care--failure to do so, as a friend of mine learned last year when I had to help him update a year old install, can result in an upgrade path that's nearly impossible. Fortunately, ARM was still up at the time.

There certainly is something to be said for stable distros with a strict upgrade path.

> I wonder if the advantages could have been accomplished without the binary file format.

I don't know. I'll be honest: I can't think of a time when I've actually made use of the filtering features of journalctl and systemd's binary format. Nearly all circumstances that have required me to examine the log are usually done by observing the last few entries or manual searches. To that extent, I'd agree that it bears a certain sense of overkill.

That said, it is still possible to feed syslog-ng entries from systemd. So I can't think of a reason to fret much over specific tooling or the likes that expects text logs.

IMO, the udev/systemd relationship, repo, etc. is probably a much more valid complaint, especially for distributions that aren't planning on using systemd in the near future as it likely has some impact on developer/maintainer workload. Although from a sysadmin perspective, the benefits of systemd are very real and material, at least in my experience.

Service introspection could probably use some work to simplify, so that's another complaint. I can never remember the commands half the time. ;)

Re:What should we expect from a modern init system? by mixmastamyk on Monday January 20, 2014 @01:52PM (#7091123)
That sounds handy, but I wonder if json could provide the same thing.
Re:What should we expect from a modern init system? by bkor on Monday January 20, 2014 @02:18PM (#7091255)
It is binary so you can quickly filter (indexes). E.g. on service, but any field is possible. You can output it in json. systemctl gives you the last lines of service output exactly because of the indexes + binary file. Using json for that just because "OMG binary" seems strange. Json is not readable.
Re:What should we expect from a modern init system? by mixmastamyk on Monday January 20, 2014 @08:13PM (#7092769)
Postgres can query inside json. To be honest I don't care that much about logging as long as it is saved to disk.
Re:What should we expect from a modern init system? by rodgerd on Monday January 20, 2014 @04:51PM (#7091984)
I am not a fan of binary logs, either.

I am a fan of more consistenly structured/filterable logs that are easier to pin to their service. I'm an even bigger fan of the fact systemd goes to a more Unixy model of not making every daemon author re-invent the logging wheel for their double-forking daemon.

Re:What should we expect from a modern init system? by brokenparser on Monday January 20, 2014 @05:11PM (#7092067)
Bull, we've had a syslog facility for ages.
Re:What should we expect from a modern init system? by rodgerd on Monday January 20, 2014 @05:48PM (#7092224)
Which would be why pretty much ever web server and mail server choose to implement their own logging mechanism, right?

Or perhaps its that having to bundle your logs together into a comparative handful of categories and then sort through them is less than entirely optimal.

Re:What should we expect from a modern init system? by brokenparser on Monday January 20, 2014 @06:22PM (#7092349)
Daemons implementing their own logging in addition to syslog is a feature, not a bug. With journald, even more so.
Re:What should we expect from a modern init system? by dredmorbius on Saturday January 25, 2014 @03:50AM (#7120308)
My understanding is that for a number of services (postgresql comes to mind) the issue was that syslog isn't implemented on all platforms, notably Windows.
Re:What should we expect from a modern init system? by jamesmiller5 on Monday January 20, 2014 @02:27PM (#7091306)
I too was put off at first when Arch moved from rc.conf to systemd as I found rc.conf simpler to use, though I trust there was a good reason to move as they are pretty good at managing the distribution.

Rachel mentioned why binary logging makes sense for complicated logs that require rather tricky state machines/regex. Apache for example, in her case.

http://rachelbythebay.com/w/2012/02/10/logs/

Re:What should we expect from a modern init system? by spystath on Monday January 20, 2014 @01:19PM (#7090950)
Systemd is a decent init system, fast (ssd+systemd=10 sec from button press), and dependency resolution is quite good. The units are configurable in the /etc/systemd directory and take precedence over the (/usr)/lib/systemd units. It is not perfect but it works. I believe it is somewhat better that sysvinit especially in dependencies. You just need an After=foo.service or Wants=foo.service and systemd takes care of everything. I know this is supported by sysvinit but the whole numbering thing always seemed like a hack to me. Systemd gets it right.

That being said, I can't say I am a fan of the integration of other services within systemd, it seems somewhat bloated. And I certainly don't like the whole journal thing. When the journal becomes bigger it becomes considerably slower. And I don't get the point for binary logs. I mean, logs are just text, why overcomplicate things?

I've been using it for over a year in Arch it is really good as an init daemon, I believe it is definitely a step to the right direction especially for desktops/workstations, probably for mobile as well. The rest of the package though, I am not so sure.

Re:What should we expect from a modern init system? by antihero on Monday January 20, 2014 @12:20PM (#7090578)
It's quite trivial to override services - you just cp /usr/lib/systemd/system/myservice to /etc/systemd/system/myservice and edit it there.

This has the advantage of not having you screw up the actual package managed files, too.

Re:What should we expect from a modern init system? by rodgerd on Monday January 20, 2014 @04:49PM (#7091975)
No, you don't copy it! Source it ( . /usr/lib/systemd/system/myservice at the head of /etc/systemd/system/myservice ) then override the lines you want to change. Much easier for upgrades.
Re:What should we expect from a modern init system? by antihero on Monday January 20, 2014 @07:51PM (#7092674)
Neat! Thanks for that.
Re:What should we expect from a modern init system? by chousuke on Monday January 20, 2014 @01:41PM (#7091073)
The "systemd" project really isn't about having just an init system. systemd by itself is an init, but the project includes several other tools related to managing the system as a whole beyond just starting services. The project integrates a bunch of utilities that each accomplish something on their own.

To me, it makes perfect sense for these tools to be tightly integrated, both with each other and with the OS kernel, simply because they are interrelated and integration leads to greater stability and a less fragile system overall.

No doubt upstream could be less hostile towards attempts to distill out the portable bits, but I can understand their viewpoint... Systemd is ultimately about making Linux-based systems run well.

Re:What should we expect from a modern init system? by sparkie on Monday January 20, 2014 @03:46PM (#7091645)
> simply because they are interrelated and integration leads to greater stability and a less fragile system overall.

This is completely counter to the way we actually design software - which is to create modular, loosely coupled components, using abstractions which allow them to adapt or be replaced, and trying to make as few assumptions as we can, since we can't predict the future. It's well understood that tight coupling is what leads to fragility and maintenance nightmares, due to decades of experience writing and maintaining such abominations.

Re:What should we expect from a modern init system? by chousuke on Tuesday January 28, 2014 @06:11PM (#7140516)
That just sounds like an argument from idealism. Writing software is all about tradeoffs, and not supporting non-Linux systems is one tradeoff that systemd makes, for a good reason.

In this case, close integration with the kernel is required to be able to produce the features envisioned.

You might perhaps find some instances where systemd components are needlessly interrelated, but the claim that it isn't modular is mostly a myth.

Re:What should we expect from a modern init system? by Elhana on Monday January 20, 2014 @01:06PM (#7090870)
If Lennart Poettering were building a bicycle, it would have multiple seats, 4 wheels, engine, conditioner, bar, sat tv, console and small pool on it. You would be able to remove most of it tho... maybe. If you try to remove the engine, it would complain that modern all bicycles are broken without engines, but you can do it at your own risk.
Re:What should we expect from a modern init system? by bkor on Monday January 20, 2014 @02:07PM (#7091202)
Your comparison makes no sense at all. E.g. systemd dependencies are minimal.

My take: if he builds a bicycle it would be lean, fast and he'd analyse all existing designs to make that happen.

Re:What should we expect from a modern init system? by Sssnake on Monday January 20, 2014 @06:05PM (#7092293)
What do dependencies have to do with anything? If I make a huge massive bloated mess of shit, it isn't suddenly "lean" because I included everything in it rather than linking to it as a dependency.
Re:What should we expect from a modern init system? by bkor on Tuesday January 21, 2014 @09:37AM (#7095433)
I suggest investigating systemd then. It isn't bloated. See for instance http://people.debian.org/~stapelberg/2013/06/09/systemd-bloa.... There are a lot of different parts to it.

Note the time of the blog. Saying systemd is bloated is IMO so 2013 :P

Re:What should we expect from a modern init system? by asdasf on Tuesday January 21, 2014 @09:54AM (#7095561)
It is bloated. Saying "I am cool with how bloated it is" does not change that.
Re:What should we expect from a modern init system? by kephra on Monday January 20, 2014 @05:11PM (#7092066)
> not to have everything symlinked to /usr.

an init system should be able to run !BEFORE! /usr is even mounted. Thats the division between /directory and /usr/directory, that /usr is mounted later.

Re:What should we expect from a modern init system? by Shish2k on Tuesday January 21, 2014 @08:01AM (#7094942)
While that's the theory, it hasn't actually worked in practice on most distros for many years -- rather than having "core stuff in root, extra stuff in /usr", we now have "core stuff in initrd, some extra stuff in root, different extra stuff in /usr, with different distros splitting the latter two up differently", the usr merge actually brings us back to the original clean design of having one "core" place and one "extras" place
Re:What should we expect from a modern init system? by midas007 on Tuesday January 21, 2014 @07:11AM (#7094765)
And udev, which was folded into systemd.