If you want the rebind to fail only when the socket is bound by another process, it may be the other process is about to close the socket, and your atomic-rebind would result in the socket not bound by either process.
That outcome is identical to the outcome from the available method, where you attempt connect() first and only bind-and-rename if the connect() fails with an appropriate error. When there's no race, you always get the desired outcome of the socket bound by exactly one process, but if the processes are racing, the result can be the socket not bound by either process.
Because both methods produce the same outcome in the race case, and the only difference is an unobservable difference in each process' logical clocks (you can't tell which events really happened first), both methods actually have the same race properties.
If you want to always end up with exactly one server running, and to always avoid taking away the socket from an existing server which is running just fine, I think you need to involve some protocol. You can try connect() and then if it succeeds, send a request "are you shutting down?". An appropriate type of connect() error or "yes" means it's safe to bind a new socket and rename over, otherwise leave the socket alone.
To atomically unlink an existing socket and bind another in place, without races, you should be able to first bind a new socket to a temporary name in the same directory, and then use rename(2) to atomically replace the existing name.
There may be queued connections that need accept(2) in the old socket after the rename, but that's not a race condition.