Make sure you're not bringing in superfluous dependencies. Use nativeBuildInputs (like devDependencies) when appropriate rather than buildInputs (like dependencies).

Can I override stdenv to not pull in GNU stuff, but instead busybox. Or none at all? Stuff like that makes the image really small…

stdenv shouldn't end up in the container at all. the container should literally just contain your derivation and its transitive dependencies. your app, its actual dependencies, and nothing else. it won't even contain bash unless you or something you reference adds it. stdenv stuff is just used to build, it doesn't end up in there. maybe use one of the nix tools to show the actual dependencies you're bringing in and cut the fat.