Yes, that was exactly the idea behind s6-fdholder-daemon: set up a central server to keep fds open when you need to restart a process. The old process stores the fd into the fdholder, then dies; the supervisor starts the new process, that retrieves the fd from the fdholder, and starts serving.

And if you don't want to use a supervisor or a fdholder, you don't even need to coordinate, and you never need to fork: simply re-exec your executable with your serving socket in a conventional place (stdin is good). Daemons should be able to take a preopened listening socket and serve on it; hot-restarting is then a simple matter of one execve(). There's really no reason to make it more complicated.