News for hackers. Stuff that matters.
Re:Linux from Scratch by hollerith on Saturday October 05, 2024 @03:09PM (#41752154)
NixOS separates packages. If the package foo contains a file /usr/bin/foo, NixOS installs it in /nix/store/67c25d7ad7b2b64c67c25d7ad7b2b64c-foo/usr/bin/foo. In order to make this separation work, Nix must sometimes rewrite binaries so that all references in the binary to /usr/bin/foo becomes references to /nix/store/67c25d7ad7b2b64c67c25d7ad7b2b64c-foo/usr/bin/foo.

The advantage of this approach is that it gives more control to the distro maintainers and the admins of the computer, taking that control away from the "upstream" maintainers of the software being packaged. For example the software being packaged cannot just call the library bar because bar is not at /usr/lib/bar.so like it is in most Linux distros -- it is at /nix/store/17813e8b97b84e0317813e8b97b84e03-bar/usr/lib/bar.so, but of course the software does not know that unless the person creating the Nix package (the packager) arranges for the software to know it (again sometimes by doing a search-and-replace on binaries).

Note that if this separation of packages imposed by NixOS has any beneficial security properties, it is merely security through obscurity because there is nothing preventing a binary from covertly searching through the directory listing of /nix/store/ for the name of the library it wants to call. Nevertheless it turns out the be useful to seize some control away from upstream in this way even if technically upstream could seize the control back if it were willing to complicate the software to do so.

People, including the creator of Nix and NixOS (Dolstra), will tell you that NixOS's main advantage is "declarativeness" (which in the past Dolstra called "purity") or the fact that the compilation / building process is deterministic. I believe they are mistaken about that. Specifically, I believe that being able to rely on the build process of a package to be deterministic is useful, the separation I described is much more useful to most users and prospective users of NixOS.

Re:Linux from Scratch by _ank_it on Saturday October 05, 2024 @03:39PM (#41752352)
Extremely noob here, I was trying nixos and got real confused about how to install python packages as pip was not allowed at system level.
Re:Linux from Scratch by hebocon on Saturday October 05, 2024 @03:53PM (#41752445)
I'm new as well - it's good to remember that NixOS, nix-shell, and the programming language of Nix are all separate. You can start with your current distro to learn nix-shell first.

I still have no idea how it all works but it seemed prudent for me to at least try.

Re:Linux from Scratch by j0057 on Saturday October 05, 2024 @04:59PM (#41752849)
If other distros allow pip-installing into the system, that could be considered a bug or at least an anti-feature, because it's almost always a bad idea: it can clash with distro-managed Python packages, it will break on Python upgrades, and sooner or later you will run into version conflicts (a.k.a. DLL Hell). Recent versions of pip refuse to install into the system by default, for all of these reasons.

It's better to instead pip-install Python packages into virtual environments, recent Pythons havr `venv` built in for this purpose. For user-scoped or system-scoped utilities, `pipx` can manage dedicated virtual environments and symlink them into the search path.

Re:Linux from Scratch by imiric on Saturday October 05, 2024 @05:14PM (#41752959)
Nix purists would say that you should use flakes to declare all the dependencies for each project, and reference all Python dependencies as Nix packages there. Nix effectively tries to replace every package manager in existence, so all Python, Ruby, Emacs, etc. dependency trees are duplicated in Nix.

I think this is insane. Not only will many packages be missing from Nix, you will also have to wait for the upstream changes to actually propagate to Nix repositories. This all assumes, of course, that there are no packaging issues or incompatibilities in this repackaging.

This is one of the ways that Nix sometimes just gets in your way. I've been using Nix(OS) for several years now, and this still bothers me.

Instead of doing this, For Python specifically I would suggest installing pyenv, which Nix packages. Then enter a nix-shell with a derivation thingie[1,2], and install Python as usual with pyenv. Then you can use any Python version, and with pyenv-virtualenv (which Nix _doesn't_ package...), you can use venvs as you're used to. Sure, you don't get the benefits of the declarative approach and isolation as with "the Nix way", and you may run into other issues, but at least it's a workflow well known to Python devs. Hope it helps!

[1]: https://gist.github.com/imiric/3422258c4df9dacb4128ff94d31e9...

[2]: It took me way longer than I would like to admit to figure this out... This shouldn't be so difficult!

Re:Linux from Scratch by therein on Saturday October 05, 2024 @05:38PM (#41753139)
Ironically, that's when you create an Arch container in LXC.
Re:Linux from Scratch by hollerith on Saturday October 05, 2024 @08:17PM (#41753796)
I would try installing pacman in the same filesystem NixOS is in.
Re:Linux from Scratch by imiric on Saturday October 05, 2024 @05:43PM (#41753168)
Meh, maybe. Containers make everything more difficult for local development, though. That would be my last resort in this case.
Re:Linux from Scratch by nextos on Saturday October 05, 2024 @06:14PM (#41753315)
The right way is to compose the base Python package with the libraries you want. For example, this gives you an ephemeral environment with the latest python3 plus NumPy:

  nix-shell -p 'python3.withPackages(ps: [ ps.numpy ])'
In case of Python, you can also go for a simpler option that avoids composing Python with its packages, but that gives worse isolation:

  nix-shell -p python3 python3Packages.numpy
If other packages were present in that ephemeral environment, aside from python3, they could see NumPy in a global directory. That's why the first option is preferable, as it offers better isolation. In Nix, some languages only let you use libraries with something like the first option. See the Nix user wiki for further information: https://nixos.wiki/wiki/Python.
Re:Linux from Scratch by rcxdude on Sunday October 06, 2024 @06:23AM (#41756139)
The correct answer, whatever distro, is to either use the system package manager or use python's venvs. Mixing multiple package managers in a given area of the filesystem is a recipe for breakage.