News for hackers. Stuff that matters.
Re:We're migrating many of our servers from Linux to FreeBSD by Spivak on Monday January 24, 2022 @12:15PM (#30060631)
You have to do it for correctness. Every use case needs this handled.

Okay so you have a PID file somewhere /var/myservice/myservice.pid. The contents of that file is a number which is supposed to correspond to a process you can find in proc.

But PIDs are recycled or more likely your PID files didn't get cleaned up on reboot. So you look at your file and it says 2567, you look up 2567 and see a running process! Done right? Well it just so happenes that a random other process was assigned that PID and your service isn't actually running.

pidfd's are the real real solution to this but the short and long tail of software uses pidfiles.

Re:We're migrating many of our servers from Linux to FreeBSD by fullstop on Monday January 24, 2022 @12:29PM (#30060853)
> But PIDs are recycled or more likely your PID files didn't get cleaned up on reboot. So you look at your file and it says 2567, you look up 2567 and see a running process! Done right? Well it just so happenes that a random other process was assigned that PID and your service isn't actually running.

If you're unlucky, though, pid 2567 might match another myservice instance. This can easily happen if you're running many instances of the same service. Even checking /proc/$PID/exe could give you a false positive.

Re:We're migrating many of our servers from Linux to FreeBSD by dataflow on Monday January 24, 2022 @04:26PM (#30064281)
I don't expect many programs do this (and I agree the real solution would be handles) but it should be possible to check the timestamp on the PID file and only kill the corresponding process if its startup time was earlier.

There might still be race conditions but this should cut down the chance dramatically.

Re:We're migrating many of our servers from Linux to FreeBSD by capitainenemo on Monday January 24, 2022 @12:25PM (#30060793)
How about keeping pidfiles on tmpfs mounts so they do get cleaned up? I guess that'd be an organisational thing to get all the apps to change to a consistent /var/tmp so you could mount it..
Re:We're migrating many of our servers from Linux to FreeBSD by fullstop on Monday January 24, 2022 @12:43PM (#30061072)
That's still not good enough. pids wrap and eventually it could point to something valid, especially on a busy system.

systemd uses linux cgroups so that it knows exactly which pids belong to the group.

The defaults have surely changed over the years, but pid_max used to be ~32K by default. On the system I'm typing this comment on, /proc/sys/kernel/pid_max is set to 4194304.

Re:We're migrating many of our servers from Linux to FreeBSD by capitainenemo on Tuesday January 25, 2022 @10:20AM (#30072737)
Wait... how would pid wrapping impact anything at all unless the process died and the pid file was stale (which is usually a fairly unusual circumstance - I've had it happen maybe a dozen times over the last couple of decades). And I guess if you have stuff surprise dying without any of the usual cleanup, you have a pretty serious situation in general. Maybe need a process monitor service just for that :)

I was focusing on the "surprise startup" scenario where it would all get wiped and reset.

Re:We're migrating many of our servers from Linux to FreeBSD by fullstop on Tuesday January 25, 2022 @10:42AM (#30073025)
As with anything else, your chances increase with the number of processes you have running.

If it blindly reads the pid and sends a kill signal, your odds are pretty good with a limit of 32K pids on a busy system. If it confirms that the process name matches an expected value, you have less of a chance.. but if your process name is bash, java, or python, maybe not as good as you would hope.

I don't have things crashing a lot, but it's naive to pretend that it never happens. It could result in two things: the rc system telling me that everything is fine or the rc system sending signals to some unrelated, poor, unsuspecting, processes.

I don't need a process monitor just for that. I have systemd. :-)

Re:We're migrating many of our servers from Linux to FreeBSD by cesarb on Monday January 24, 2022 @05:50PM (#30065199)
> The defaults have surely changed over the years, but pid_max used to be ~32K by default. On the system I'm typing this comment on, /proc/sys/kernel/pid_max is set to 4194304.

The commit which changed the defaults is this one: https://github.com/systemd/systemd/commit/45497f4d3b21230756...

Re:We're migrating many of our servers from Linux to FreeBSD by lmm on Monday January 24, 2022 @09:49PM (#30067278)
How does systemd make sure it never reuses a cgroup, OOI? If you're worried about PIDs wrapping, surely that applies to anything randomly-generated as well.
Re:We're migrating many of our servers from Linux to FreeBSD by fullstop on Monday January 24, 2022 @09:52PM (#30067308)
It's derived from the service name, which is guaranteed to be unique. It's similar to how names need to be unique in /etc/init.d/