Scaling Mastodon with systemd template units

Re:Scaling Mastodon with systemd template units by bkq on Tuesday November 29, 2022 @06:10AM (#33786086)

systemd, in my opinion (one that doesn't seem to be widely shared), really does make administering a Linux server easier. No longer do I have to wrangle with shell scripts for the process control of daemons, or worry about the dependencies between them, instead I can organise that logic in a nice and neat .service file for systemd to worry about. The templating feature as this article demonstrates is also incredibly handy for DRYing up some .service files you may write.

Another feature of systemd I really like are the timers [1][2] it introduces. It offers a much more sane approach to handling scheduled jobs, and allows for an easy overview of which jobs are running and when the next job will fire via systemctl --list-timers.

[1] - https://www.freedesktop.org/software/systemd/man/systemd.tim...

[2] - https://wiki.archlinux.org/title/systemd/Timers

[ Parent | on HN ]

Re:Scaling Mastodon with systemd template units by arnsholt on Tuesday November 29, 2022 @07:41AM (#33786859)

I think it's more widely shared than one might believe, because of the people who don't like systemd there's a contingent that really, REALLY hate it.

[ Parent | on HN ]

Re:Scaling Mastodon with systemd template units by ilyt on Tuesday November 29, 2022 @08:26AM (#33787268)

I despise the amount of random stuff that's tied to it and journalctl data format is still an abomination (and my bug with it trying to search thru entirety of log dir to show single app status is still open)

...but we did remove few thousands lines of fixed init scripts (anyone that tells you sysv scripts are easy and simple is lying to you, multiple projects fail there) *and* an bunch of Monit instances thanks to the features the base service management has. And simplified a bunch of other cases.

So yeah, overall even with issues it is a huge benefit.

[ Parent | on HN ]

Re:Scaling Mastodon with systemd template units by necrotic_comp on Tuesday November 29, 2022 @09:23AM (#33787921)

The process control in systemd I think is unambigously good, but it would be nice if it wasn't as tightly coupled with journalctl and all of the other random stuff it's gobbling up.

[ Parent | on HN ]

Re:Scaling Mastodon with systemd template units by ilyt on Tuesday November 29, 2022 @10:16AM (#33788555)

Well, to be fair you can't exactly ask typical syslog demon "give me log files of an app" but the way journalctl implemented it is just fucking terrible.

It takes 4 whole seconds [1] for systemctl/journald to tell me it has no logs for the daemon on my NAS and it opens 985 files while it is doing it

If it just used sqlite as a backend it might've been useful for analysis (on top of way faster...) but Lennart wanted to have a go at implementing binary DB format badly so we're stuck with it.

If it at least kept a pointer to last file the app's logs were written to the lack of proper indexing also wouldn't be a problem.

But nope, it's just entirely worse than text format. And I do mean that in entirety, acking thru text files is faster than systemctl trying to find log files for the app...

* [1] https://github.com/systemd/systemd/issues/2460

[ Parent | on HN ]

Re:Scaling Mastodon with systemd template units by taskforcegemini on Wednesday November 30, 2022 @04:25PM (#33807945)

My favorite experience with journalctl from many years ago was when an application kept spamming the logs till there was no space left. Once that point was reached, the binary logs got corrupted so couldn't be opened to investigate the cause. Don't remember how many turns it took me.

[ Parent | on HN ]

Re:Scaling Mastodon with systemd template units by chrismonsanto on Tuesday November 29, 2022 @09:02AM (#33787660)

systemd is a significant improvement over script-based inits (and upstart, which I always found buggy). Nobody sings its praises because they don't have to: it "won" a long time ago.

[ Parent | on HN ]