Rez ( https://github.com/nerdvegas/rez ) is used by some visual effects companies (including mine) to manage software packages.

It allows us to have great flexibility in the mix & match of software versions and runtime environments.

For Nix, this is almost a side-effect, however. Nix's primary task is to describe how to build packages, and the entire OS is just another package that groups all dependencies. You can make a single change to a file everything depends on and it will result in a new OS.

Nix can evaluate the entire system configuration in seconds, and build or download missing binaries in parallel.

As a result you can e.g. have a complete OS based on a different glibc (maybe with some patch you like) installed and running alongside the normal OS without the glibc patch. Packages that do not use glibc are simply shared.

>"Using Rez you can create standalone environments configured for a given set of packages. However, unlike many other package managers, packages are not installed into these standalone environments. Instead, all package versions are installed into a central repository, and standalone environments reference these existing packages. This means that configured environments are lightweight, and very fast to create, often taking just a few seconds to configure despite containing hundreds of packages."

  - A depended on version X of C
  - B depended on version Y of C 
  - X != Y

You can't have A and B in a same runtime environment obviously.

At different runtimes (e.g: different package.py configurations) yes.

I can have a software called my_application which has a version that uses A and another version that uses B.

That differentiation can be a flag at runtime, a "variance" e.g: gcc-4 vs gcc-3 etc.

How does nix allow you to do that? For example, say I am using library foo and it has the function bar

In version 1

bar definition is

```

def bar(a, b):

    return a + b

In version 2

bar definition is

```

def bar(a):

    return a + 10

I don't understand how you can use v1 and v2 of foo at the same runtime? Unless nix does some namespacing based on the lib version invisibly for the enduser but that looks very prone to error...

If you release them as separated libraries it won't complain, for example, shotgun API imports can be by version

```

import shotgun3

import shotgun2

```

so theoretically you can release a rez package whose name is shotgun3 and another one whose name is shotgun2.

That way you would effectively have shotgun2 and shotgun3 in a same runtime environment.

Nix expends a bunch of complexity into fucking with dynamic linking to make that work transparently most of the time and deploying wrapper scripts to cover other cases, so I was curious if rez has a cleverer solution there.

you have build_requires which you can list dependencies that you require at build time (e.g: gcc-3 vs gcc-4) but not at runtime.

So you can have things that were built with gcc-3 and gcc-4 running side by side in a same shell environment without issues.

gcc itself, in this case, would be a rez package that can have multiple versions installed in the system.

So Nix is ultimately a tool for making and sharing reproducible package builds. It has a binary cache, but it's not necessary. Like Ports, packages get built by default.

Docker, on the other hand, is a distribution and execution mechanism. It provides an abstract way to move around a fully assembled, ready-to-go service or application running in isolation.

It's entirely reasonable to use both. You can use Nix to build and manage docker images and make extremely minimalist docker images. You can use Nix knowing that the entire process is perfectly reproducible, and the Docker containerization is only a final integration step.

With this, you sorta get the best of both worlds. You get a reproducible build (and if done right, also a reproducible dev environment via nix-shell) and with Docker you get the ability to build and run a prepped copy with a well-defined interface.

Docker really doesn't provide a way to reproduce a built image from scratch. You sort of have to trust and build on existing images, and most folks making bulk images appeal to external tooling outside docker files to do this.

NixOS committer here.

Docker attempts to achieve reproducibility by capturing the entire state of a system in an image file. It also attempts to conserve space by taking a layered approach to images, so when you base your Dockerfile on some base image, your resulting image is the union of the base image's layers and your own changes.

Here's where Docker's approach falls down, and how this could be fixed (and indeed is, by Nix).

Flaw #1:

Building an image from a given Dockerfile is not guaranteed to be reproducible. You can, for example, access a resource over the network in one of your build steps; if the contents of that resource changes between two `docker build`s (e.g. a new version of whatever you're downloading is released, or an attacker substitutes the resource), you'll silently get different resulting images, and very likely will run into "well, it works on my machine" issues.

Solution:

Prohibit any step of your build process from accessing the network, unless you've supplied the expected hash (say, sha256) of any resulting artifacts.

For the projects I work on in my free time, using NixOS on my personal computers, I've never been bitten by nondetermism.

I wish I could say the same about my work projects that use Docker. My team members and I have run into countless issues where our Dockerfiles stop working and then we have to drop everything and play detective so e.g. new hires can get to work, or put fires out in our C.I. env when the cached layers are flushed, etc. So many wasted hours.

Flaw #2:

What happens if you have two Dockerfiles that don't share the same lineage, but you install some of the same packages? You end up with multiples layers on your disk that contain the same contents. That's wasted space.

Solution:

I'll use NixOS as an example, again. In NixOS, you can look at any package and compute the entire dependency graph. It should be noted that this graph includes not only the names of the packages, but also precisely which version of each package was used as a build input. This goes for both build inputs and runtime dependencies.

Note: by "version" I mean not only the version as listed in the release notes, but every detail of how the package was built: which version of Python was used? And then transitively: what version of C was used to compile that Python? etc, etc.

NixOS exploits this by allowing you to share packages with the host machine, and then each NixOS container you spin up has the necessary runtime dependencies bind-mounted into the container's root file system. As a result, NixOS has better deduplication (read: zero). Also, by the same graph traversal mechanism, it's trivial to take any environment, serialize the runtime dependency graph, and send that graph to another machine, back it up somewhere, create a bootable ISO for CD/USB -- whatever you can dream up.

Thanks to Nix's enforced determinism, you can trivially build container environments (and all of their required packages) in parallel across a fleet of build machines. In fact, Nix's superiority at building packages is so strong that people have gone so far as to build Docker images using Nix instead of `docker` (where they can't avoid Docker entirely for whatever reason): https://github.com/NixOS/nixpkgs/blob/2a036ca1a5eafdaed11be1...

You can read more about the Docker image building support here (along with my own rationale for this in the comments): http://lethalman.blogspot.com/2016/04/cheap-docker-images-wi...

I'm keeping things simple here and trying to address the most salient "Docker vs Nix" points, though I could continue talking about other strengths of NixOS outside of the scope of Docker/container tech, if desired.

Docker's union filesystem approach is great in a world where you can't use a better package manager. For everyone else, there are package managers that obviate the need for such hacks, don't waste space, provide determinism at both runtime and build time, etc.

Nix isn't only a package manager, it is also a functional programming language intended for system administration. This means that, while a Nix file is comparative to a Dockerfile, it has several key differences:

1. All Nix files are just functions that take a number of arguments and return a system config (like a JSON object, but with some nice functionality). A Dockerfile is a set of commands you run to build a system to a "starting" state. This is imperative (you're telling the computer to "do this", then "do that", etc.), so once it's done, you can mutate state to deviate from what you specified in your Dockerfile. With Nix, while you can technically do this on some systems, it does provide you with the command line tools so you don't break things (e.g. nix-shell, nix-env). Note, on NixOS, other measures are taken to encourage safety.

2. If things go wrong in Nix(OS), the idea is that you can do a fresh install in your system, copy your old Nix file to it, and with one bash command, be back to where you were before things went haywire. In terms of containers, there's nothing new with this because this is exactly what Docker does. However, Nix also has this concept of generations, so every time you use a Nix command to change your system state either declaratively via a Nix file or imperatively in the command-line using nix-env, you can roll back to a previous version of state. This is especially nice with NixOS, because it creates generations for your entire system too (includes hardware config, drivers, kernel etc.), and makes separate GRUB entries for each generation, so if something breaks after you do a system upgrade, you just chose an old GRUB entry to go back to where you were. AFAIK, Docker doesn't offer anything like this, and this is a good example of how these tools' designs can impact their feature-set so dramatically.

3. A neat feature of Docker is composability. You can inherit from other, pre-existing Dockerfiles, and you can deploy multi-container apps with various tools. Composability at a single-container level is very straightforward with Nix. Since every config is just a function, you simply call the function exposed by a different Nix file with the correct arguments, and... voila! Once you've made your desired Nix file, you can run it either using nix-shell or nixos-container. While I'm no expert, I believe they perform better than Docker as they don't use virtualization. For multi-container deployment, there is NixOps. You write some Nix files describing the VMs you want to deploy, and run a bash command to deploy them to various back-ends (AWS, Azure, etc.). Again, the big difference here is that you can incrementally modify these VMs in a safe way using Nix. If you change your deployment config file, Nix will figure out something has changed, and modify the corresponding VMs to achieve the desired state.

Some may believe that Docker and Nix are very similar, and to their credit, they are in certain scenarios. The thing I like about Nix is that it's one language (and architecture) that was designed well. It's minimal, yet makes it possible to do so much in a safe way.

Nix has been around for a while, but I think the community is growing quickly as functional programming continues to take off. I'm excited to see where it goes, and am super grateful I have a tool like this to use while coding.

This is a problem with lots of feature-rich software, even with meticulously-documented APIs. What we need is reverse-indexed documentation. That is, an extensive API reference is only useful for someone who already knows what functions are in the API and just needs to remember how to use them. But even the most thorough API reference does nothing to promote discovering new functionality. This is often left to the authors, who then have to go about writing a User's Guide that gradually explains concepts, idioms, etc. in prose.

Thorough User's Guides are rare because they are tough to write, and even tougher to write well. Users don't often have the time to read through potentially hundreds of pages of prose to find what they're looking for. We need a better way to let users search or browse for concepts, and then be given a list of the functions that implement each concept.

That is, addition to documentation like:

    size_t strlen(const char * s);
      RETURN: Length of string s.

    size_t strnlen(const char * s, size_t maxlen);
      RETURN: Length of string s, or maxlen (whichever is smaller).
      NOTE: Stops reading after maxlen.

    char * stpcpy(char * dst, const char * src);
      Copy src to dst.
      RETURN: pointer to trailing '\0' of dst, or dst[n] if no trailing NUL.
      NOTE: Undefined behavior if dst and src overlap.

    char * stpncpy(char * dst, const char * src, size_t len);
      Copy up to len bytes from src to dst.
      RETURN: pointer to trailing '\0' of dst, or dst[n] if no trailing NUL.
      NOTE: Undefined behavior if dst and src overlap.

    char * strcpy(char * dst, const char * src);
      Copy src to dst.
      RETURN: dst.
      NOTE: Undefined behavior if dst and src overlap.

    char * strncpy(char * dst, const char * src, size_t len);
      Copy up to len bytes from src to dst
      RETURN: dst.
      NOTE: Undefined behavior if dst and src overlap.

    strcpy
      TAGS: "concept":"data type":"text", "concept":"attribute":"length",
            ".input":"array", ".input":"char", ".input":"pointer",
            ".return":"string", ".return":"pointer"
    strncpy
      TAGS: "concept":"data type":"text", "concept":"attribute":"length",
            "concept":"data type":"array":"max-length operator",
            ".input":"array", ".input":"char", ".input":"pointer",
            ".return":"array", ".return":"pointer"

    concept
    └─ data type
       └─ text
       └─ array (conceptual)
          └─ max-length operator
    └─ attribute
       └─ length / size
    data structure
    └─ char
    └─ array (implementation)
    └─ pointer
    

    '"concept":"data type":"text" AND "concept":"attribute":"length"'

It seems like a PITA at first, but I'm pretty sure tagging functions in their docstrings is easier than writing a whole new User's Guide.

Looks like the Nix expression language is untyped, so this wouldn't work directly, but maybe adding a rough type signature in the docstring would get some of those benefits (and it should be a bit better for discover-ability, since you wouldn't need to guess the same tags/concept the author choose).

[1] - https://www.haskell.org/hoogle/

The way it works is you write a sequence of arguments and an expected result, and it suggests a method to call.

For example, I just now tried it, entering

  3. 4. 7

  3 + 4 --> 7
  3 bitOr: 4 --> 7
  3 bitXor: 4 --> 7

Another example: input of

  'hello world'. #('hello' 'world')

  'hello world' substrings --> #('hello' 'world')

  '  abc  '. 'abc'

  '  abc  ' asLegalSelector --> 'abc'
  '  abc  ' withBlanksTrimmed --> 'abc'

It's a total hack, of course, but none the less effective or useful for that.

It has a list of methods marked "safe to experiment with", and simply tries them out.

It gets a big boost from being able to evaluate the receiver (the first in the input list) to a concrete object, and then only consider methods on that object's class.

So how did you know what to search for? A simple case was to look up the ready-built application's GUI which usually always contains some strings. So if you wanted to change something in the system which would result in a change to its GUI just search for source-code with some of the words you saw in the GUI.

In Smalltalk everything is an object, that is including numbers, characters etc. The standar method-lookup searches from the innermost class methods to the outermost class methods.

The method finder could simply iteratively go through all applicable methods that return the class requested and match results.

[1]: http://esug.org/data/Old/vw-tutorials/vw25/vw25ug.pdf (page 42 method-lookup)

One neat trick I'd forgotten is that if it doesn't find anything interesting with the inputs in the order written, it permutes them and tries again!

I also think that you could get a lot of people to agree that it is a great idea, and STILL have a lot of trouble enforcing it in a project without very rigid code review policies and very good linting rules.

Isn't all that also true for Guix?

If so, why do your prefer Nix over Guix?

Unfortunately, that isn't workable for most of us.

There may even be community-based sources for non-free packages, but they are explicitly ignored by the Guix project.

I haven't missed any non-free packages since switching to Guix surprisingly, I always seem to find a free alternative though my daily req are pretty basic.

Where?

> The community won't help anybody if they ask for help with non-free software though.

This, specifically, is where Nix has the advantage.

I cannot answer your question, but I think Nix has (for some reason) a lot more publicity than Guix at the moment.

But if "derived" implies that Guix is a fork, then you are mistaken.

Guix is not a fork of Nix.

This is a GNU-approved fork

> weakly motivated

Has lots of strong motivation, like a stricter stance on non-free software, the usage of a "real", more expressive language (Guile) instead of a niche, poorly undocumented DSL (Nix), use of GNU Shepherd instead of SystemD for the init system, among other

> niche NIH rehash

It shares most of the codebase with Nix, so to say it's NIH is missing the point. Nix improvements are shared with Guix, and Guix improvements are shared with Nix

> unless you're already victim to lisp-induced stockholm syndrome

Different lisp offer different syntaxes, features, and use-cases. That you cannot see it makes me think you didn't try too much, and are probably just trolling.

In Guix, the only code from Nix is the Nix daemon. There is not really much code going back and forth, mostly just bug fixes.

But the package manager itself might be a good idea for dev environments, but all the languages i use provide similar facility(not counting system-library dependencies).

At that time, most packages were old version on nixpkgs and some weren't available. I will look into it again, when i have the mood to switch distros.

Many things I got from AUR packages are binary-cached in stable Nixpkgs, not to mention "compilers" (GCC, clang, and various versions of GHC in my case). I don't know when you last tried it out, but binary availability isn't a problem now if you'd like to give NixOS a spin. The only thing I build from source is polybar[0].

[0]: https://github.com/jaagr/polybar