You're describing a "distributed filesystem with client-side whole-file caching". The Andrew File System (AFSv2) did this back in the 1990s.

https://pages.cs.wisc.edu/~remzi/OSTEP/dist-afs.pdf

FWIW, complexity is introduced when the server is unavailable and the client-side caches need to be synced with the remote.

Sshfs, NFS, and SMB are substantially easier to set up and live with, so you're not fighting conflict resolutions inherent in multi-primary repositories.