Scaling Mastodon with systemd template units

Re:Scaling Mastodon with systemd template units by ilyt on Tuesday November 29, 2022 @08:46AM (#33787473)

    Environment="NODE_ENV=production"
    Environment="PORT=4000"
    Environment="BIND=::"
    Environment="STREAMING_CLUSTER_NUM=2"

You probably want to use EnvironmentFile= and shove that file with the rest of app's configs

And, uh, shove the queue size parameter there instead of

    sudo systemctl enable [email protected]
    sudo systemctl enable [email protected]
    sudo systemctl enable [email protected]

using feature designed to run multiple instances as a poor man's way to configure stuff. Let's just say nothing will stop you from also doing

    systemctl enable [email protected]

and systemd will just try to start the sidekiq twice

The article's proposed way of using templates is TERRIBLE. I had assumed from the title it will show off how to run multiple instances using same template units (say to vary a run dir or whatever), but no, it is using them as glorified config file for number of threads/db conns. Just shove that into env file, you can use env parameters in rest of the file too...

[ Parent | on HN ]

Re:Scaling Mastodon with systemd template units by arminiusreturns on Tuesday November 29, 2022 @09:22AM (#33787901)

You sound very knowledgable on systemd, could ypu share some suggestions for learning resources? I would like to push a better use of it in my org, but besides some unit file messing about and using all the cool anaylysis tools, I don't know as much as I'd like.

[ Parent | on HN ]

Re:Scaling Mastodon with systemd template units by ilyt on Tuesday November 29, 2022 @09:52AM (#33788285)

Well, I've written a blogpost on dependencies [1] but really reading all of the man pages (the most useful ones [2][3][4]) will get you a good idea on what it can do, I had many "oh, we can use THAT" moments with it. They are written pretty well.

The directives we got most use in org are probably Restart* family and RequiresMountsFor, althought the last one is more and more common in distro packages so it is not even needed that often. The usecase being "don't start DB if mount fails" usually.

Overrides are probably the most used feature overall, we often find that the standard distro unit just needs a little bit of tweaking for our purpose (like adding restart), so we just create file /etc/systemd/system/servicename.service.d/01-restart.conf with

    [Service]
    Restart=always
    RestartSec=30

and voila, service now auto-restarts and that behaviour won't break when package updates, as we're not changing any package file. The few other modifications of existing ones included adding new read-write dirs or adding env variables for proxy.

Technically you can even go as far as making services into little containers with all the options for limiting permissions but I haven't got that far. Most apps don't exactly need it either, have readonly/readwrite dir + some limits and it is pretty safe out of the box, I've also noticed many distro packages going that way.

* [1] https://devrandom.eu/blog/post/2022-02-27_increasing_system_...

* [2] https://www.freedesktop.org/software/systemd/man/systemd.exe...

* [3] https://www.freedesktop.org/software/systemd/man/systemd.uni...

* [4] https://www.freedesktop.org/software/systemd/man/systemd.ser...

[ Parent | on HN ]