Looking at Linux’s sendfile [1], it should be possible to use it like that in principle (but not with the standard http.FileServer, I guess).

[1] http://man7.org/linux/man-pages/man2/sendfile.2.html