News for hackers. Stuff that matters.
Re:The Magic Nix Cache, a GitHub Action for speeding up your... by Karupan on Monday June 26, 2023 @07:17PM (#36487111)
As a complete nix noob, will this help with caching node dependencies? We have a few projects that take over 20mins for a `yarn install && yarn build`. I’ve read setting up Nix for node isn’t that straightforward, but that was a couple of years back. Has anything changed with respect to node projects?
Re:The Magic Nix Cache, a GitHub Action for speeding up your... by biggestlou on Monday June 26, 2023 @07:29PM (#36487212)
You may find something like node2nix helpful (https://github.com/svanderburg/node2nix). This converts your package.json into a Nix expression that can then be cached. You're right that it does require some setup and a bit of Nix knowledge but could yield significant benefits and take a good chunk out of that 20 minutes.

Another option might be to use pnpm instead of Yarn and cache your pnpm dependencies. pnpm actually works a bit like Nix in that it creates a pnpm-lock.yaml file with content-based hashes for the full package.json dependency tree. This enables it to quickly determine which parts of the dependency tree it needs to build and which are already available.

Re:The Magic Nix Cache, a GitHub Action for speeding up your... by soraminazuki on Monday June 26, 2023 @09:52PM (#36488079)
Nowadays, there's buildNpmPackage [1]. It's included in Nixpkgs, actively maintained, and easier to work with IMO.

[1]: https://github.com/NixOS/nixpkgs/blob/master/doc/languages-f...

Re:The Magic Nix Cache, a GitHub Action for speeding up your... by duped on Monday June 26, 2023 @07:49PM (#36487361)
In order to support resolving dependencies to multiple versions (eg A and B depend on C but at incompatible versions), what npm does is drop the version override for C inside the nested node_modules directory (node_modules/A/node_modules/C, for example).

That means that node_modules (as created by a package-lock.json) can't really be cached or be built from caches since it depends on the particular version solution found by npm for a particular project.

So there's only so much that Nix can do. It can cache it about as well as using a naive caching scheme with actions/upload-artifact or similar (create a tarball of your node_modules and just cache it across runs, update when you need to).

Basically node_modules is inherently large. If you want better performance for caching dependencies use a better programming language environment.

Re:The Magic Nix Cache, a GitHub Action for speeding up your... by crabmusket on Tuesday June 27, 2023 @04:39AM (#36490402)
Or fewer dependencies!
Re:The Magic Nix Cache, a GitHub Action for speeding up your... by maxloh on Monday June 26, 2023 @08:02PM (#36487474)
That is supported officially:

  - uses: actions/cache@v3
    with:
      # npm cache files are stored in `~/.npm` on Linux/macOS
      path: ~/.npm
      key: ${{ runner.os }}-${{ hashFiles('**/package-lock.json') }}
Re:The Magic Nix Cache, a GitHub Action for speeding up your... by wereHamster on Tuesday June 27, 2023 @12:48AM (#36489031)
I have given up installing node dependencies via Nix. I use Nix to make nodejs available but then install node deps with npm/yarn/pnpm.

Node packages sometimes pull additional files from the internet in a postinstall script, or do other funky stuff that's incompatible with Nix. So the idea that you can construct a pure derivation from a package-lock.json or yarn.lock file is a pipe dream.