I believe mp4 files can be repackaged to put the bookkeeping data at the front of the file, which makes them playable while doing a sequential download.
Re:JPEG XL: How It Started, How It’s Going
by Tigress8780 on Thursday July 20, 2023 @10:41PM
(#36809408)
That metadata is usually put around the end of the file for compatibility reasons, but one can use ffmpeg's `-movflags faststart` option to move it to the beginning (very common in files that are meant to be served on the web).
HTTP file transfer protocols support partial downloads. A client can choose just to not to download irrelevant audio. I think most common web platforms already work this way, when you open a video it is likely to be in .mp4 format, and you need to get the end of it to play it, so your video player gets that part first. I am not entirely sure.