I've had luck with using the approach in here https://nixos.wiki/wiki/Python#Emulating_virtualenv_with_nix... because it allows the escape hatch of using pip if something isn't packaged and I want to quickly try something out. I have also been meaning to try mach-nix which may obviate the need for pip at all. Having a good standard skeleton for a python/nix project would be super helpful to add to the wiki, etc.

I tried multiple solutions myself, this one is the closest to what I want, because:

- overlays the packages on top of nixpkgs - understands setup.cfg and can get dependencies from it

It is still not perfect, but when I find some time I think to submit a PR to nixpkgs to improve setupcfg2nix. Particularly it doesn't understand tests_require which you can list dependencies to run unit tests. Nix has equivalent checkDepends.

Unfortunately, the security updates introduced in MacOS Catalina have made installing Nix on the latest version of MacOS a fairly involved process.[1] I looked at using it when I rebuilt my computer recently and decided to wait until it was more baked.

[1] Catalina restricts which directories can be added to the root dir. Nix uses /nix.

There is a way to deal with the latest round of developer-hostile changes from Apple:

https://github.com/NixOS/nix/issues/2925#issuecomment-604501... documents the workaround but if you don't have a direct link it's almost impossible to find because GitHub's UI is terrible.

The problem here really seems to be with nix. The /nix path shouldn't be hard coded. (Security concerns aside, I don't want a package manager to clutter up my root directory.)

It's necessary to hardcode paths in order to ensure that the exact version of a dependency is linked into the executable. Without this mechanism, nix packages would not be reproducible and self-contained.

Also, the path isn't really hardcoded. You can use a custom path, but that means you can't use binary caches.

I'm sure there are reasons for it, but it still sucks.

It ought to be possible to use binary caches and still swap out the path. (Modifying a string in an ELF executable isn't that hard.)

It's not just ELF executables that hardcode the path. There's a ton of different ways to run software which all use hardcoded paths. Which is good! That's the entire value proposition of Nix, in a way, that it's now safe to use hardcoded paths. But you can't edit all those different hardcoded paths in all those different filetypes.

Where else are paths hardcoded? This seems like a huge mess.

Just picking one example off the top of my head, pkg-config files.

Modifying a string in ELF is hard. As long as the string has the same length as the original one, it has a good chance to work (but still fails for compressed data). But if you want an arbitrary directory, you need to replace it with a different length path, and that will probably break a lot of things.

The nix author created utility called patchelf which is used by nix all the time during the build (to rewrite library paths).

As far as I know it creates a new elf each time it is called (it doesn't modify a file on place) so the length of library name is not an issue.

It's not that hard if you leave some spare space for string constants in the .rodata section. Nix already patches ELF executables as things stand.

The problem is how it affects all the hashes, which aren't recalculable in such an easy way.

Could you not check the hashes before doing the substitution?

The issue is just fixed! https://github.com/NixOS/nix/pull/3212

This is only sort of true: I’ve found that Nix on a Mac is great except that about 10% of the derivations just don’t work: some are excluded explicitly as broken and some just fail in strange ways and need lots of debugging.

For me, most of packages worked, the ones that didn't were actually justified to not work, definitively far more than 10% (it was more like 90% for me). The stuff that didn't work was justifiably linux specific, except one thing, that now memory escapes me what it was.