You're looking at a solid 10-20 hours of hacking stuff together to get a decent working system, and you still won't understand a good 80% of what you copied from various repos.

I've been using NixOS for a year now, and I have a definite love/hate relationship. Many times I miss the simplicity of Arch, where I understood _everything_ about my system and how it was configured.

NixOS is like learning everything all over again, but in a weird, poorly documented language+standard lib that nothing else uses.

I think when you are super comfortable with it, is when it becomes truly amazing. But that takes a lot of dedicated time and effort.

Yes, it's extremely powerful once you truly understand Nix.

There are efforts to improve documentation, but it still is lacking (I think the biggest problem is that Nix is so big, not just the OS but it can be utilized as a build system).

Just with NixOS is not exactly clear how can you for example build your custom image.

I think https://nix.dev/ is approaching the documentation from the right direction.

There are also many pieces that people built that you need to find.

For example some things that I found accidentally:

* https://github.com/nix-community/poetry2nix

* https://github.com/matthewbauer/nix-bundle

* https://github.com/cleverca22/not-os

Unfortunately those side projects often have even worse documentation.

If you want to build custom images just use https://github.com/nix-community/nixos-generators It's documentation is on point.

Would it be possible to add a minimalistic image that contains app, kernel and essential packages? Like what you would get when building a docker container or using the not-os repo that I mentioned earlier.

I think having mini images would be awesome.

You might like: https://nixery.dev/

e.g. if you want a Nix container with a shell environment, git, and htop:

    docker run -ti nixery.dev/shell/git/htop bash

This is where I'm sitting right now. I'm trying out Void Linux on my laptop and it's just so simple. I love that I understand my system. I look at Nix as a language and I just have no idea what is going on.

What I'd really like, as a learning resource, is a "Nix for apt/yum users" or a "Nix for Ansible" users guide. There is the cheat sheet (https://nixos.wiki/wiki/Cheatsheet) but it's just not enough.

For example, I wanted to just... create an arbitrary directory in my home directory (~/workspace). Does Home Manager do this? If so, how? Do I just have a shell script that is called within a package? That seems excessive. Is this not a use case for Nix? It looks like it can do arbitrary tasks across the system but the documentation is so bogged down with theory that it takes forever to find practical applications.

> For example, I wanted to just... create an arbitrary directory in my home directory (~/workspace). Does Home Manager do this? If so, how? Do I just have a shell script that is called within a package? That seems excessive. Is this not a use case for Nix? It looks like it can do arbitrary tasks across the system but the documentation is so bogged down with theory that it takes forever to find practical applications.

You are definitively trying to do too much at the same time. Before using Home-Manager (that is a community project, it is not endorsed by NixOS except by allowing to be hosted on the nix-community group on GitHub), I stayed at two years playing with only NixOS. It is only after I got comfortable with Nix and NixOS that I decided to migrate my configuration to Home-Manager. Not saying that you need that much time to use HM, just that HM assumes that you're know a reasonable good amount about Nix and NixOS.

So I would say, start with less: try NixOS first, it is kinda easy to use compared to other "advanced" Linux distros like Arch and Void. The easy things are really easy, the hard things can be really hard at first but I don't think you need to know much about Nix to be proeficient on NixOS nowadays. Also yeah, you will need to search some things, but since the scope is reduced it is easier to find answers.

My advice would be (per the other person here) to ignore home-manager when you're getting started, and then, when you're ready, treat it as a dotfiles manager on steroids.

I've never used it to manage visible directories in my ~/, but for ~/.gitconfig? Absolutely :).

Using NixOS with flakes is actually quite simple once you already know how to do it, but the lack of high quality tutorials made it difficult for me to get started. I hope the community will improve this situation over time.

Could you write one?

> how painful was it to learn and get working?

nixpkgs (which you can use on macOS or any Linux distro), quite little, you can get up to speed in 15min as a Homebrew replacement:

    nix-env -qas ruby             # 'q'uery 'a'vailable 'search'
    nix-env -I ruby.              # install by package name (not recommended)
    nix-env -iA nixpkgs.ruby_3_0  # install by "attribute", recommended
    nix-env -q                    # 'q'uery (i.e list installed)
    nix-env -e ruby               # uninstall (like deleting a git ref)
    nix-env -q
    nix-collect-garbage           # cleanup, like `git gc`
    # etc...

And as a general purpose virtualenv/rvm/whatever replacement:

    $ cat shell.nix
    {
      pkgs ? import <nixpkgs> {}, # you can have multiple of these with different names to have e.g a mix of stable and unstable
    }:
    let
      openssl = pkgs.openssl; # just an example of how to set a var
    in pkgs.mkShell {
      buildInputs = [
        openssl
        pkgs.ruby_3_0 # or reference attires directly
      ];
    }
    $ nix-shell
    (nix) $ ruby --version
    ruby 3.0.1p64 (2021-04-05) [x86_64-darwin17]
    (nix) $ openssl version
    OpenSSL 1.1.1k  25 Mar 2021
    (nix) $ exit
    $ openssl version # back to the system one
    LibreSSL 2.8.3

That way it's immediately useful and you can dig in deeper into the concepts as you go, if you want to.

NixOS, it's barely different, only generalised to the whole OS, but it's a bit tougher, because there's this abstraction via configuration that generates e.g systems files or other configurations that you'd operate with directly otherwise on a traditional distro. But the minute you screw things up, having the generation selector to pick and boot on straight in the boot loader you can see the absolutely unparalleled value of the proposition.

Please don't recommend that people use nix-env. It's one of Nix's biggest footguns and a huge support burden for the maintainers.

We've been actively trying to remove mentions of it from the documentation.

If you want something "installed" use home-manager. If you just want something for quick dev use nix-shell.

I'm so glad this is the official line now. People giving examples using nix-env -I is, IMO, one of the major reasons it's so hard to get up and running.

It's like... here are all the amazing reasons to do declarative config... and here's how you do everything using nix-env -I. Good luck figuring out how to translate it into a NixOS config so you can get all the benefits of declarative config that we just described!

> Good luck figuring out how to translate it into a NixOS config so you can get all the benefits of declarative config that we just described!

Guix has a command that converts an imperative profile to a declarative one. Check out the `--export-manifest` option for `guix package`: https://guix.gnu.org/manual/en/guix.html#Invoking-guix-packa...

Adding this to nix-env's replacement would be a sound approach imo. Maybe we could have one that also emits a little message for each package which is used in a NixOS module, directing users to a `programs.whatever` option instead of just dumping it in their `environment.systemPackages` list

If nix-env is a big ol' footgun, are there any plans to make home-manager a part of the OS itself, instead of just a community project?

My hope is that when Sander van der Burg finishes drafting his RFC for mainlining his process management framework [1], which abstracts over ways to manage local services from systemd to supervisord to Docker, we can actually unify the module collection in Nixpkgs with its clones in nix-darwin and home-manager, and offer a more complete Nix experience as a configuration/service manager rather than just a package manager on non-NixOS.

Fwiw, you can already have nixos-rebuild read the home-manager configs for all of your users and deploy them as part of the normal NixOS config update process using the included NixOS module.

——

1: https://github.com/svanderburg/nix-processmgmt

Random nix usage question... occasionally I run into something like this, where the "install" directions are to clone a git repo and then install in an imperative way. What's the correct way to do this declaritively, e.g. in a home-manager or NixOS config? And why are directions like this in the imperative form? I'd think that people working this closely to core Nix would be pushing declarative setup so there must be something I'm missing.

Another example is nixops 2.0.

I think this is one of the things that flakes aims to solve. I'm playing around with a new flake-based setup and I'm using flake-utils-plus[1], and I use the overlays builder to make it easy to refer to packages that come from flake inputs in my configuration.nix and similar places, like. I just added the latest Nixops from master to make sure I was giving you a real answer, and I use an overlay like this in my flake.nix:

      (final: prev: {
        nixops = inputs.nixops.defaultPackage.${prev.system};
      })

after adding the flake input for nixops:

      nixops.url = "github:NixOS/nixops";

to my system flake's `inputs` attribute. And after a `nixos-rebuild switch`, nixops reports the highly following mysterious version number :

   nixops --version
  NixOps @version@

If you're not using Nix flakes, you can do something similar with Nix and Nixpkg's fetchers as well, also using overlays, using the `nixpkgs.overlays` option in NixOS. This is how, for example, the community Emacs overlay recommends folks use it[2].

What you wanna do for projects that don't offer a flake.nix or a ready-made overlay is to use a fetcher like in the Emacs example, but write your own overlay function that invokes Nixpkgs' `callPackage` function on whatever Nix expression inside the repo represents the package you want, or imports them from a release or default.nix as appropriate. Home-manager uses this to define its own little overlay[3], the one that gets used in its flake.nix, and its package is just its default.nix[4].

Unfortunately, in the pre-flakes world, you just have to read the Nix code and figure out how to translate things to get the attributes you want into scope. For example, this works for nix-processmgmt:

      (final: prev: {
        nix-processmgmt-tools = (import (
          (builtins.fetchTarball "https://github.com/svanderburg/nix-processmgmt/archive/6def8584c6b028c922c550859a07b989d21d6f73.tar.gz")
            + "/tools/default.nix")
          { pkgs = prev.pkgs; }
        );
      })

and then you can add, e.g., `nix-processmgmt-tools.common` to your `environment.systemPackages`.

I think maybe the reason instructions aren't given for some projects is that they see those parts of the guide as mostly for beginners or casual experimenters, and they expect advanced or ‘serious’ users to be able to figure it out without much trouble. To some extent I think this is because different people choose to pin packages in the pre-flakes world through a variety of different mechanisms, and the authors of these packages and tools don't know in advance how you want to do it.

In the case of nix-processmgmt, I think Sander doesn't actually expect to have any users! In such cases, an imperative install that users are expected to play with for a little while and then just throw away is supposed to be enough.

You're right, though, that it's odd and disappointing that instructions for the preferred way of doing things are sometimes simply not given. A polite pull request or issue report would probably be well-received. My recommended strategy, if you get stuck, would be to ask for help getting those packages into scope declaratively on Discourse or Matrix, and then to offer the authors of these out-of-tree packages pull requests to modify their READMEs accordingly. :)

PS: You don't necessarily _have_ to use overlays for these. You can also drop expressions that use builtins.fetchTarball and then use `callPackage` or import from those sources directly into lists of packages like `environment.systemPackages`.

—

1: https://github.com/gytis-ivaskevicius/flake-utils-plus

2: https://github.com/nix-community/emacs-overlay#quickstart

3: https://github.com/nix-community/home-manager/blob/master/ov...

4: https://github.com/nix-community/home-manager/blob/master/de...

Wow, a lot to digest here, thanks for the super detailed response! Definitely going to be coming back to this over the next week or two and trying some things out.

> Unfortunately, in the pre-flakes world, you just have to read the Nix code and figure out how to translate things to get the attributes you want into scope

This has been my approach, I always kind of assumed that there must be a better way but at least I feel a little better knowing that there isn't really. I haven't spent much time with flakes because I mostly only use Nix as a package manager/operating system but this helped me see how even if I'm not personally writing packages that I want to re-use I can still get a lot of value from flakes, so getting that setup is probably my next step!

I don't think removing it is a good idea. Nix is a paradigm shift and things are done completely different, nix-env though is still one piece that might be somewhat similar to what people are used to.

I like to use nix-env (or more often, now, `nix profile`) for a persistence level in between nix-shell and really adding something to my home-manager or NixOS config. I let my profile build up 10-20 things installed, then every few weeks I decide what belongs in my declared config and uninstall everything in the profile.

Imo entirely removing imperative package management would be a mistake, although imperatively managing a file that gets sourced in your declarative config (a bit like /var/dpkg/selections on ol' Debian) instead of putting the whole profile manifest in the Nix store and leaving it at that would be better.

> I let my profile build up 10-20 things installed, then every few weeks I decide what belongs in my declared config and uninstall everything in the profile.

I'm curious what you get out of this that you don't get from just adding/removing packages in your home-manager config? Is it just a matter of it being quicker to do nix-env -iA instead of updating your config and running home-manager switch? Or is there some other benefit?

I do have that use case as well. Taking a parallel from git, it's kind of a working/staging area/stash. There's a feeling of zero commitment. It could be I try out some unknown package, or that I have a temporary one-shot need that still lasts longer than a nix-shell -p.

But mostly I use it as a beachhead for people to eventually jump ship, helping them climb the first step of the ladder:

"see? It's easy to install and use, you'll be quite autonomous. And if you feel it's not your thing just ignore it or rm -rf /nix"

Then I bait them with e.g shell.nix just enough to tease their curiosity.

So instead of feeling overwhelmed by a whole new arch and contractually tied by configs they feel free, empowered, and curious.

> Is [the benefit of this approach] just a matter of it being quicker to do nix-env -iA instead of updating your config and running home-manager switch?

That's definitely a factor. I think since I keep my Nix configurations in source control, somehow modifying the configuration feels more ‘official’, and it also usually comes with extra steps like committing and pushing.

The other thing I like is that it makes it very easy to tell if I actually want/need something: if I find myself installing something over and over (because I periodically purge my profile), I know for sure that it's time to add it to my config. This way I end up pulling less crap I don't actually use into my setup in an enduring way.

Maybe I'd also feel the same way about invoking things via `nix run` or `nix-shell` over time, and that would motivate me to incorporate them into my config ‘for realsies’ by declaring them.

> instead of updating your config and running home-manager switch?

I'm not currently a home-manager user on NixOS. Before home-manager was a thing, I used to define groups of packages using buildEnv and store them in an overlay for very simple de-facto declarative package management, so something like:

  nix-env -iA pxc-tui-apps

would install my whole CLI environment at once, and then on NixOS, I'd include `pxc-tui-apps` in my `environment.systemPackages`.

I'm switching to home-manager in my new setup, but one thing I still like about `nix profile install ...` (the flakes-based/next-gen replacement for nix-env, currently) is that it's user-mode/unprivileged, and it doesn't involve rebuilding my whole system (or anything other than the dependencies of just the package I want), even if my nixpkgs checkout/channel/flake registry or whatever has changed. `home-manager rebuild switch` is also unprivileged and also doesn't involve updating my whole system, but unfortunately home-manager doesn't support flakes just yet. You can use it on flakes-based setups on NixOS and macOS via the home-manager NixOS and nix-darwin modules, respectively... but then you're giving up the other benefits I like, because you have to invoke nixos-rebuild after all!

If `nix profile` were some day removed along with `nix-env`, but I had a user-level declarative environment management tool (like home-manager or something more tightly integrated), I could probably get by with just a little discipline about how I choose to edit my configurations and manage sources of Nix expressions and be pretty happy.

But I think the problems with `nix-env`/`nix profile` are pretty solvable, and I think lacking any imperative solution at all will likely put some ‘winnable’ new users off.

I do agree that `nix-env` itself sucks and needs to go, though. `nix-env --upgrade` doesn't really do what people expect, and there's no real reason to use/prefer it over just removing/reinstalling a package. The way that `nix-env` thinks about versions is basically insane, since Nixpkgs doesn't have any real version metadata and `nix-env` just parses attribute names to get the versions back out. `nix-env --query` is clunky and slow (but the new `nix search` is awesome and crazy fast!). `nix-env` was a cool thing for its time, and it's actually how `home-manager` manages its profiles on the backend (which is why flake support is still lacking; enabling flakes disables `nix-env` for your user). But it's like an imitation of `rpm`, and it was made before Nix had a real userbase and opportunities to think about what operations/abstractions/metadata were desirable at that level.

One of the things that's cool about Nix's design is that its design allows you to just bypass the hardest and most annoying problem that faces traditional package managers: dependency resolution. If you want a package manager that's guaranteed to give you solutions that are correct and complete, you need a SAT solver for dependency resolution, and that's NP-complete. By leveraging its quasi-content-addressable derivation approach, Nix gets to avoid resolving dependencies like traditional package managers do— the thing you need is the thing you were built against, and that's that! Similarly, by leaning hard into the Nixpkgs monorepo so that almost all packages live on it and all third-party package sources are built as de facto overlays on top of it, Nix has been able to totally avoid having to think about versions.

Compared to formats like DEB or RPM, Nix packages have very, very little metadata. Nix packagers don't have to declare things like acceptable version ranges for dependencies, what packages provide, what package names ought to be considered equivalent, what other packages it's incompatible with, or even what version number a package has. But `nix-env`'s `-q`, -i`, and `-u` options all imitate the `rpm` CLI, where all of that kind of metadata is necessary and present. And `nix-env` makes all that stuff work by assuming the structure of Nixpkgs and operating on Nix attributes representing packages directly. And it doesn't really make sense in the Nix world as it exists.

Flakes is the first attempt to revisit all these questions the community has basically punted on like ‘how do we want to relate packages to each other in a way that's not monorepo-centric?’, ‘what kind of metadata do we actually want to have for publishable Nix source package artifacts?’, ‘how should Nix code repositories advertise what features/tooling (packages, shell environments, modules, overlays, whatever) they support?’, ‘do we want Nix to actually be able to reason about versions?’, etc. (This also possibly reintroduces the question of dependency resolution. Hopefully not?) I think once we have real, considered answers for questions of that kind, grounded in the experience of the community so far, we can build an imperative frontend to Nix that makes sense and is nice to use.

> or even what version number a package has

In the context of within nix packages that's correct. In the context of nix usage that is not.

I start a project, I need it to use Ruby 2.7.x, OpenSSL 1.1, node 14.x because that's what the project is compatible with, and I need that to happen on both Darwin, Linux, on whatever cpu arch. Pinning the hashes of "whatever I built it with" won't work.

Worse, some software such as e.g Ruby encodes their platform at build time (because it matters, because #ifdefs) so currently I'm on darwin20 but specifying "ruby" pulls in RUBY_PLATFORM==darwin17. Nix is currently helpless in face of that.

> and it doesn't involve rebuilding my whole system

True. One thing I'm 100% sure is that nix-env -i will do just that and nothing else, whereas nixos-rebuild switch might include something else pending I might have forgotten about because it relies on globals.

> I start a project, I need it to use Ruby 2.7.x, OpenSSL 1.1, node 14.x because that's what the project is compatible with, and I need that to happen on both Darwin, Linux, on whatever cpu arch. Pinning the hashes of "whatever I built it with" won't work.

> Worse, some software such as e.g Ruby encodes their platform at build time (because it matters, because #ifdefs) so currently I'm on darwin20 but specifying "ruby" pulls in RUBY_PLATFORM==darwin17. Nix is currently helpless in face of that.

Right, right. That's a real problem. Gentoo Portage has mostly a big monorepo kinda like Nixpkgs, and in it you can find multiple versions of many pieces of software. Maybe the sensible future in the Nix world would be:

1. Our package attributes include version metadata, perhaps through something like ‘subflakes’ within the repo.

2. Nixpkgs includes multiple versions of major pieces of software.

3. Nixpkgs' top-level remains basically as it is now, in that for the most part only the latest version of something is used as a dependency. Alternatively, we use some kind of a ‘lock file’ that gets published with Nixpkgs. This way `nix profile install` still doesn't have to perform any dependency resolution for packages inside nixpkgs, so its behavior stays fast and predictable.

4. You can add version constraints in defining packages for use outside Nixpkgs, in `nix shell` environments, etc.

Does that seem like the way to go for you, or is your ideal picture something else?

> If you want something "installed" use home-manager.

I hear you. I understand nix-env as it exists needs to go, and I can only trust you on the support side. I did not know about home-manager.

Doing my homework, from home-manager README, I can read:

> Before attempting to use Home Manager please read the warning below.

> Unfortunately, it is quite possible to get difficult to understand errors when working with Home Manager, such as infinite loops with no clear source reference. You should therefore be comfortable using the Nix language and the various tools in the Nix ecosystem.

> Home Manager targets NixOS unstable and NixOS version 20.09 (the current stable version), it may or may not work on other Linux distributions and NixOS versions.

> Also, the home-manager tool does not explicitly support rollbacks at the moment so if your home directory gets messed up you'll have to fix it yourself.

On top of being third party (for now?), all of this really does not bode confidence in the tool and severely raises the bar for adoption when all one wants is to install tmux globally for their user (IOW a bunch of symlinks in ~/.nix-profile/bin that "just works").

But the best one is this in the manual:

> This manual will eventually describes how to install, use, and extend Home Manager.

There is no section in the manual describing the usage of the tool. The Getting Started seems to be solely about development and contributing. The terse examples in the README is what made me search for the manual, and I could only get a grasp of what they entail because I have NixOS experience. Seriously, if this is deemed "general public availability" quality, this is borderline user hostile.

So, I do note the envisioned deprecated-ness of nix-env but I will continue to use that as a first rung to help people climb the ladder when I introduce people to nix until there is a suitably accessible and reliable replacement.

Indeed I successfully used it as a beachhead for people to eventually jump ship:

"see? It's easy to install and use, you'll be quite autonomous. And if you feel it's not your thing just ignore it or rm -rf /nix"

Then I bait them with e.g shell.nix (which feels like Gemfile) or nix-shell -p --run (which feels like docker run) just enough to make it relatable while teasing their curiosity.

So instead of feeling overwhelmed by a whole new arch complete with a foreign config system and an alien language, and contractually tied by configs they feel confident, uncommitted, free, empowered, and curious, and that much more likely to transition to the next rung up.

> If you just want something for quick dev use nix-shell.

That I 100% agree with, which is why I immediately described it as well. It's the hook to the next rung.

> In some cases Home Manager cannot detect whether it will overwrite a previous manual configuration.

That, to me, is a problem. I fully acknowledge the limitations and caveats of nix-env, and duly highlight them when introducing people to nix, but this means home-manager is not the tool I need, in git parlance it's way too much "porcelain" and not enough "plumbing". It feels invasive when you're still in the process of getting acquainted.

I do not want home-manager to handle launchd, nor .gitconfig. I do not use nix-darwin or NixOS, _on purpose_, as I use nixpkgs as an additional tool, an extension of existing systems.

I am fine if nix-env has some of its features revamped, or is deprecated and replaced by a better tool but in terms of some of its existing use cases it is exactly the tool needed, i.e merely allow users or root to make a package persistent and widely available for the user or for root in their PATH, as if it were a native package, and with no other side effects.

I hope you understand these are legitimate needs.

I use nix-shell for dev environments. Once I've got it working it's wonderful - full replicable, across multiple development machines and CI. It's painful though. Some problems I've encountered recently on different projects:

Old version of fwup packaged. Can override to get newer version, but needed to add another dependency to make tests pass (and it gets built locally rather than being cached)

Pain getting my environment to include the right Python packages so I can run the gigalixir command line tool.

Have to workaround an oddity to do with how Erlang is packaged so that I don't get lots of warnings when anything is compiled:

    shellHook =
    ''
      export ERL_LIBS=""
    '';

Just a hint - extra attributes in the `mkShell` appear as environment variables in the built shell. So instead of a `shellHook` for this you can just have an attribute `ERL_LIBS="";`

I'm not completely sure but I think this approach is preferred because shellHook only runs in `nix-shell` but if you ever wanted to `nix-build` a release it might not run.

There tools like direnv, that "enter" your env environment as soon as you cd to the directory. This is implemented by changing environment variables like PATH.

Because shellHook allows on executing any code, there's no standardized way to undo what it executes, so those tools don't execute that part.

My experience, a while ago I will admit, is that your experience will vary wildly based on what you’re doing with it.

Writing Haskell in it was a joy, but Ruby was pure pain. I actually never could get RoR to run (this might have changed), so I gave up and went to OSX for that stuff.

I haven't used Ruby, but Python also was a huge pain and there were many attempts at fixing it, but all had some issues for me. That is until I found poetry2nix.

My understanding is that Python packages are directly installable via nix, at least some of them. This is a completely parallel way of installing packages from the regular way, which sucks, but at least it works in theory. I never personally tried it though.

Stack (Haskell) was awesome because it’s Nix aware, enabling it to create its own env to handle C deps on it’s own.

How does poetry2nix compare to mach-nix, in your opinion? The way mach-nix handles native (non-Python) dependencies strikes me as really slick.

So I haven't used mach-nix, but from what I see it provides its own way of specifying dependencies. While in case of poetry2nix you are using poetry for dependency management and poetry2nix just translates it to Nix.

The biggest benefit for using poetry is the dependency resolver which is much smarter than in pip and actually finds packages that match the version requirements (although looks like mach-nix also has a smart resolver.

Other benefits are:

* nice CLI

* ability to package application and upload to PyPI

* poetry can still be used by people who don't use Nix

Regarding non-python dependencies, this is solved by overrides file[1]. That way most of the time you don't have to think about it. You can also override the override in your project as well.

[1] https://github.com/nix-community/poetry2nix/blob/master/over...