Through a new collaboration between NVIDIA, The Nix Foundation, and Flox, Nix CUDA packages are now available.

https://flox.dev/blog/the-flox-catalog-now-contains-nvidia-c...

This is a great note for AI developers, but my use case around NixOS is targetted to consumer users. The value is in self-hosted services (clouds, VPNs, etc.) without needing to be technical, and supporting the standard suite of stuff the average consumer needs/uses. (Word processor, email, internet, PDF, zoom)

My biggest concern here is not feature parity with the latest in AI; but in usability, or maybe irrelevance (what happened to thin clients?). My hope is that what stopped thin client adoption was just paying cloud providers forever, and that the average consumer that has a home computer can use that computer as a NAS to actually be their own iCloud / OneDrive, with the ability to deploy their 'home machine' on any laptop.

But more and more tools will use AI. What if your self-hosted e-mail server uses AI to filter spam messages, for example? Or what if the consumer wants to run a drawing program that can do diffusion-style gen-AI?

There is jetpack-nixos, if this an option, which is decently maintained and for a client of numtide we build https://github.com/numtide/nix-gl-host

So they can use the host driver in nix based services with Jetson

Why do you need to stick to just the "vendor supplied CUDA"? Is CUDA for arm64 targets only distributed via Nvidia ISOs for the Jetson targets? The download site seems to offer an arm64-sbsa option.

I thought the big barrier was the custom kernel they distribute. It was a pain to use the old Jetsons before Nvidia finally enabled modern c group support for containerization with docker. Perhaps building a kennel for their ended platforms is simpler now?