News for hackers. Stuff that matters.
Re:Stack and Nix = portable reproducible builds by smilliken on Thursday December 10, 2015 @05:37PM (#10714102)
I highly recommend Nix. We used to use apt-get, pypi, rubygems, npm, make, and others, and now it's all replaced with a single tool: nix. It's dramatically simplified builds and dependency management for us at MixRank.

Benefits:

* It's declarative, so no one has to run anything to get to the correct state when a dependency has changed. Open a new nix-shell and automatically get everything. It's impossible to get in a bad state because it doesn't touch your system.

* We know we're all running the same software because the build will fail if hashes don't match.

* Portable across most unix's, and windows support is in progress.

* Patching other people's code is as easy as referencing a .patch file from a nix expression. You don't have to wait for your patch to be accepted in the main project or fork the project to build a custom release.

* There's no virtualization of IO like in containers. It effectively has no overhead.

* We can easily use different versions of the same piece of software in parallel; they don't collide because the hashes don't match.

* It has a very active community. The package repository, nixpkgs [1], has 73k commits, 650 contributors, and median pull request acceptance of 18 hours.

* Our software won't silently break because one of our package managers replaced a package. We maintain a fork of nixpkgs which we rebase on our own schedule.

If you've ever been frustrated with your package manager or build tools, give nix a shot!

[1] https://github.com/nixos/nixpkgs

Re:Stack and Nix = portable reproducible builds by codemac on Friday December 11, 2015 @02:11AM (#10715827)
If you're like me and have a hankerin' for something other than the nix language, check out guix!

http://gnu.org/s/guix

All of the scheme your heart could want. I run archlinux + guix and it's a magical happy land. I'm sure the same could be accomplished with nix, but I enjoy writing scheme too much!

Re:Stack and Nix = portable reproducible builds by smilliken on Friday December 11, 2015 @02:55AM (#10715922)
I've watched a few talks on guix, and there's lots to like.

I recall there's supposed to be compatibility between nix and guix, but can't remember the details. Are they going to share package expressions, have a transpiler, or something like this?

Re:Stack and Nix = portable reproducible builds by codemac on Friday December 11, 2015 @06:11PM (#10720799)
They both use the same daemon (guix-builder is a fork of nix-builder), so I think they could share packages given the hashing is the same. This hypothetical has not been implemented yet, I'd love it.

Their is an importer[0], but this is basic and just grabs the package metadata, it's up to the packager to implement the package still. Hopefully this is further automated in the future.

[0]: https://www.gnu.org/software/guix/manual/html_node/Invoking-...

Re:Stack and Nix = portable reproducible builds by codemac on Friday December 11, 2015 @02:26AM (#10715868)
Question now that I got my guix zealotry out of the way.

Do you run your own hydra? I really want to but it seems like setting one up is a pain.

How do you handle /nix/store sizes? I've found that gc get's away from me. For 1 user with very little installed, my store (including it's hardlink dedup) is 11Gigs, and I have 59 packages installed.

Re:Stack and Nix = portable reproducible builds by smilliken on Friday December 11, 2015 @03:08AM (#10715953)
I considered running hydra briefly, but shyed away from the extra complexity and operational overhead. Most things are already available in the community hydra, and building the unique things is hardly an inconvenience since it happens once per machine. Instead of using channels, we maintain a fork of nixpkgs and occasionally squash it into a git subtree of our main repo. Our git history reflects everything you could want to know to replicate a working build of our software.

I've never done a gc on my /nix/store, and have wrecklessly added things to it while experimenting. It's 25GB and has 44k (!) derivations/packages in it.

Re:Stack and Nix = portable reproducible builds by codemac on Friday December 11, 2015 @06:02PM (#10720756)
Well my store has 8.9k derivations, but that's only for 11 packages in my profile :/
Re:Stack and Nix = portable reproducible builds by nrr on Friday December 11, 2015 @02:43AM (#10715900)
How do you maintain derivations for your own software with respect to nixpkgs?

Do you have your own function that composes things and imports <nixpkgs>, along with overrides for things that you find deficient in the derivations found in nixpkgs? Do you include derivations for your own software as part of your fork of nixpkgs that doesn't make its way back upstream?

Inquiring minds wish to know.

Re:Stack and Nix = portable reproducible builds by smilliken on Friday December 11, 2015 @03:28AM (#10715989)
Glad you asked; I've been wanted to share.

The simplest approach is to just use the default nix channel. For a team and production environment, this isn't ideal because you don't control when changes are made to your dependencies. So, we fork nixpkgs and rebase on our own schedule. This is also nice because it gives us an easy avenue to contribute upstream. Changes that may have community value go in this fork, nothing private.

The fork is then squashed as a git subtree into our main repo. That way it's part of our history like everything else. You can even git-bisect with confidence in the face of many dependency changes!

In addition, we have a tree of our own nix expressions for dependencies that are unique to us or that we haven't bothered to contribute back to nixpkgs yet. Sometimes this includes copying an expression for a dependency into our tree with a patch until the bug is fixed in an upstream project.

The final result is one or many nix files reference our private nix expressions and nixpkgs subtree, intended to be used with nix-shell.

Re:Stack and Nix = portable reproducible builds by wingless on Thursday December 10, 2015 @09:34PM (#10715150)
Do you use NixOS or Nix on another distro?
Re:Stack and Nix = portable reproducible builds by smilliken on Friday December 11, 2015 @02:50AM (#10715917)
I've been using it with Ubuntu. We also have it working with OSX.

I'd love to give NixOS a try, but haven't gotten around to it. Version controlling the entire system is very compelling.