I also think that believing you have the source code to an app but being unable to build it because of missing, possibly proprietary dependencies is far more damaging to the FLOSS spirit than any amount of format difficulty.
I think it's pretty common that stuff is being downloaded during the build phase, mostly from java package hosts like mvn, jitpack, etc. There are checks in f-droid to ensure those hosts are on a whitelist of FLOSS-policy repos but the checks don't prevent any custom build logic of downloading stuff via http, cloning git repos, etc. Also while maven central does not, some of these hosts also allow takedowns of published artifacts, at least jitpack allows it and it's on the whitelist (also jitpack's policy isn't FLOSS only, only requiring that the package is on public github, which still allows for nonfree "source available" software AND jitpack doesn't have any policy about downloaded binaries during its build).
The build that F-Droid executes comes in two phases: one to obtain source code, which then gets packaged, the second phase to execute the build. Note however that the build execution phase does have network access and many builds actually do use the network to download dependencies, so its certainly not as advanced as Debian. I'd put what F-Droid is doing to somewhere between Nix OS (which doesn't have source packages at all) and Debian (whose source packages are complete so that no internet is needed).
For F-Droid, one could maybe think about running build twice: first with internet enabled but a recording proxy in between, and second with internet disabled but that proxy replaying the recorded file. That file is then published with the sources. There are formats for this, e.g. warc. It would be a partial improvement although it might still download binaries etc instead of source code and patching the source code would be also hard (tools would have to be developed to do this), hurting the FLOSS spirit.