Nix-shell scripts are ad-hoc, what nix does here is to provide an environment. Nix derivations are different and can be fully declarative and pure. Nixpkgs pinning does not mean specific nixpkgs commit pinning or pinning a specific version of a package. You can do these as well. I wonder what was the underlying reason for your condition.

Just curious, have you written anything up about these experiences with Nix? I've been learning it recently and the reproducibility angle is a huge part of why, if it's oversold then I feel like I should reevaluate how much time I'm putting into it.

Things shouldn't change if the nixpkgs version didn't change, you didn't disable the sandbox, and you don't have master branches of random repos as dependencies. If that's the case, sounds like a major nix bug.

If the sandbox is disabled (as it is by default outside of nixos I believe) then yes, you need to be very careful that there is no outside state that leaks into your build (usually through environment variables, PATH but not only - GCC is a big offender here), which sounds like what happened in your case? For archival purposes you really should be enabling the sandbox.

Thanks: it does seem like the lack of sandboxing is the major problem here. A difficulty in this sort of archiving is that we're not trying to make it so that we can build the code: we're trying to make it so that any reader in the future can. Close to zero of those readers are likely to be using nix for anything else, or to have any experience with nix (or any other approach we might take).

Our approach was to make a script for nix-shell that would put the readers in an environment where they could run the code, compiling specific (old) versions of two research software packages. But I think the problem here is that in writing the derivations here, we didn't put gcc in the build inputs. So the system's compiler ended up being used, and with the old, finicky code, compiler flags needed to change. A frustration here is that it appears sandboxing can't be enabled for nix-shell, a problem that has been known for close to a decade [2]. But in our case, if we can get the build inputs to be reasonably complete on our own, it should improve the situation for our readers in the future.

[1]: https://github.com/DNA-and-Natural-Algorithms-Group/SST-sequ... [2]: https://github.com/NixOS/nix/issues/903

You might give GNU Guix a try, combination of guix time-machine and guix shell -C should give you the exact same environment each time without fear of being affected by the rest of the system (with the obvious exception of the linux kernel).

And it seems to be popular in science circles.

> Even with nixpkgs pinning, our code failed to compile after only a few years. Outside of nixos itself, it does not appear that the build environment is the same; aspects of compilation change, and we ended up needing to change compiler flags a few years later, while expecting that we might need to again at some point.

Was this using flakes or no?

Since you mention pinning, I'm guessing not.

An experience report, even if brief, would be very valuable to a Nix/NixOS community working seriously on reproducibility.