> Service A starts writes PID to disk, lets say 123. lots of processes start and stop as it goes along and does its work. Service A crashes/stops working PID 123 gets reused by a new process SysAdmin comes and hits /etc/init.d/ServiceA restart shell script calls kill 123 which was a totally different process now not at all related to ServiceA.

I created a specialized FUSE filesystem to deal with this. Processes create PID files in it, but when they die, the filesystem automatically removes them.

Code: https://github.com/jcnelson/runfs

Nice to see a different solution than cgroups for this

The Readme is rather sparse, could you add an example how to use it from a init shell script?

I still think 64 bit non reused pids are the best long term solution. There are other pid race conditions. (Not having pid files deleted on reboot is a different issue of course).

Although the Capsicum model (in FreeBSD, slowly getting into Linux) where you can have file descriptors for processes is another different model.

(although Posix does I believe require pid_t to be an int which is an issue)

> Not starting a database before the filesystem with the database files is mounted.

This was solved decades ago with numbered init symlinks:

  K20postgresql
  S20postgresql

This was solved in the same way that assembly languages are touring complete. It's true, but it's not useful. You need a compiler to output those meaningless numbers. What happens when you install something after booting? Who starts it? On a long running system those files are useless. Which ones have you run? Are they idempotent? Systemd attempts to solve all of these problems and more.

Don't those correspond to runlevels? I can boot to runlevel 5 and not have all file systems present and working e.g. an NFS filesytem might not be available.

So I would say it has been hacked around for decades. Not cleanly solved. But I am not the best informed here so please add more details about how numbered init symlinks guarantee file system being there before a service is started.