That's what this post builds to solving at the end and in the next post - having a unit deletes the instance from a cluster before shutdown

I tend to push the metaphor of software being meant to be read and only incidentally to be run by a computer as far as I can. We are telling a story to future us or our successors. Stories have rules and it's jarring when you violate them.

There's an idea in software that's a bit like the corollary of Chekhov's gun. Chekhov's gun is about not presaging jarring story elements that will never come to pass. But it's nearly as jarring to leave important story arcs as complete surprises until the end. Producing the gun moments before the curtain goes down would be quite a WTF. We didn't know that was a possibility. That's a niche that some people occupy, but it is a niche.

Introducing things early fights with Locality of Reference, but when we're talking about things of deep, dramatic importance (like an attempted murder, or a reaper process) it's important to introduce that "character" early in the story so that people know that it exists. Failing to do so is a form of deus ex machina and we only appreciate that in very small doses.

So framed that way, I don't see a problem with having to start a killswitch while you're spinning everything up. It's there, people can see it, and know to ask questions about it.

Is this ML output?

If it's not then it's a lot of rambling nonsense instead.

Seems a few people managed to follow it.

I can follow it just fine, but that doesn't mean it isn't rambling.

Or mushroom output?

If it is, then call me a sucker because I thought it was an entertaining perspective.

This is exactly the type of novel analogy that ML can not produce because the words don’t tend to appear next to each other in any corpus.

Until now!

Indeed! Every brilliant idea anyone ever posted online can now appear in ML-spew with all the context and wit stripped away.

Can we not make this the new low-effort way of saying "this doesn't make sense to me"? I've seen it a lot lately and it's at least as insulting as a regular old ad hominem attack.

FWIW it made perfect sense to me. It's a fun and thought-provoking analogy.

If software is meant to be read, and only incidentally run, then having it in the config to only run at shutdown is still introducing it early on. As a matter of fact, I think it would be like introducing the the reaper character in the very beginning and keeping them around in every scene sitting silently in the background until it was time for their one line at the very end. Many people might not notice, but to someone who does, it would be very odd indeed.

I have the following unit file saved for that purpose:

    [Unit]
    # https://stackoverflow.com/questions/36729207/trigger-event-on-aws-ec2-instance-stop-terminate
    Description=unlink agent from remote server
    Before=shutdown.target
    [Service]
    Type=oneshot
    EnvironmentFile=-/etc/environment
    KillMode=none
    ExecStart=/bin/true
    ExecStop=/opt/service-name/shutdown-unlink
    RemainAfterExit=yes
    User=root
    [Install]
    WantedBy=multi-user.target

If I recall correctly, the KillMode=none is important as it causes the shutdown-unlink binary to escape systemd process supervision. Without it, you may deal with systemd immediately halting your shutdown unit (and killing the process) when it hits the shutdown target.

You unit races with the network being brought down, since it isn't listed as a "Before" target, FYI.

I could be mistaken, but I don't think even that would be sufficient? Before would make your command execute before the network is brought down, but it wouldn't have the latter wait for your command to actually complete.

The post shows long-running stop scripts / containers and demonstrates them delaying shutdown (not with KillMode none though)

@CGamesPlay network.target's "primary purpose is for ordering things properly at shutdown: since the shutdown ordering of units in systemd is the reverse of the startup ordering, any unit that is order After=network.target can be sure that it is stopped before the network is shut down if the system is powered off."

https://www.freedesktop.org/wiki/Software/systemd/NetworkTar...

They need After=network.target, not Before=network.target.

The shutdown order is the reverse of startup order, and they execute the payload on shutdown (ExecStop).

The "Before=shutdown.target" is superfluous, that is a default dependency (as regular services obviously need to be shut down before system shutdown).

"KillMode=none" shouldn't be necessary, unless your command leaves processes running that need to stay running (and in that case you have other problems, since those processes may not actually finish before the system is shut down). Systemd waits for your service to stop before reaching shutdown.target (per the default Before= and Conflicts= dependencies).

Since your command probably needs networking, you also want "After=network.target" to ensure your command runs before network is torn down, as covered in sibling comments.

Thanks. I ripped it from a colleague, and now I have some bugs to report.

Sadly, I think this would be child's play with SysV init.