This is what I used to think, but after twenty years of running Linux, and at least twice that many reinstalls, I very much appreciate an OS that doesn't eventually bitrot underneath me.
"Just install this one package to get CUDA working"... no. If I do that, then in two months when I want to do something else I'll find my OS is no longer capable of being used that way. And in four, when I want this specific version of CUDA working again, I don't remember how. The instructions probably wouldn't work anymore anyway.
NixOS allows me to put in down in terms of code, and the code practically never stops working. It's also immortal -- I've never needed to reinstall it. If I did, it'd be painless, but that's only ever happened in case of hardware failure.
It's the best part of Nix.
Lucky that you don't need to run NixOS as your OS to get a lot of the advantages from it then. I use debian for my boring desktop applications but have Nix available to me for when I need any e.g. development tools.