In what world do you have a machine which downloads source code to build it, but doesn't have outbound internet access so it can't download source code or build dependencies?

Like, effectively the "build machine" here is a locked down docker container that runs "git clone && npm build", right? How do you do either of those activities without outbound network access?

And outbound network access is enough on its own to create a reverse shell, even without any open inbound ports.

The miss here isn't that the build container had network access, it's that the build container both ran untrusted code, and had access to secrets.

If you don't network isolate your build tooling then how do you have any confidence that your inputs are what you believe them to be? I run my build tools in a network namespace with no connection to the outside world. The dependencies are whatever I explicitly checked into the repo or otherwise placed within the directory tree.

You don't have any confidence beyond what lockfiles give you (which is to say the npm postinstall scripts could be very impure, non-hermetic, and output random strings). But if you require users to vendor all their dependencies, fully isolate all network traffic during build, be perfectly pure and reproducible and hermetic, presumably use nix/bazel/etc... well, you won't have any users.

If you want a perfectly secure system with 0 users, it's pretty easy to build that.

> But if you require users

I'm not suggesting that a commercial service should require this. You asked "In what world do you have ..." and I'm pointing out that it's actually a fairly common practice. Particularly in any security conscious environment.

Anyone not doing it is cutting corners to save time, which to be clear isn't always a bad thing. There's nothing wrong if my small personal website doesn't have a network isolated fully reproducible build. On the other hand, any widely distributed binaries definitely should.

For example, I fully expect that my bank uses network isolated builds for their website. They are an absolutely massive target after all.

Most banks and larger enterprises do exactly this. Devs don't get to go out and pick random libraries with out a code review and then it's placed on a local repository.

There are just far too many insecure and 'typo' malware to pull off the internet raw.

Hell, even just an unrestricted internal proxy at least gives you visibility after the fact.

This is npm with all dependencies stored in a directory. Check them in. You do code review your dependencies right? Everywhere I’ve worked in the last 10 years has required this. There is no fetching of dependencies in builds. Granted, this is harder to pull off if your devs are developing on a totally different cpu architecture than production (fuck you apple).

It's common, doesn't mean it's secure. A lot of linux distros in their packaging will separate download (allows outbound to fetch dependencies), from build (no outside access).

Unfortunately, in some ecosystems, even downloading packages using the native package managers is unsafe because of postinstall scripts or equivalent.

>Unfortunately, in some ecosystems, even downloading packages using the native package managers is unsafe because of postinstall scripts or equivalent.

Funny you should mention this because I was just psyching myself up to submit my blog piece from last night on the topic.

In Python, downloading packages using the native package installer (Pip, which really doesn't itself do anything that could be called package management) is unsafe because of build scripts - unless you tell it to only accept pre-built packages, defeating the point of the systems these Linux distros are using. (I assume/hope people in this position are aware of the problem and have rigged up another solution with the API. In the post I commented that I don't know of such solutions being publicly available, but surely they exist somewhere.)

You'd be justified in wondering why the build script runs when you only ask to download the package. It's mainly because of the historically atrocious approach to metadata (and all the legacy packages for which installation is still supported). But from reading the issue trackers, it seems like the code paths aren't especially easy to disentangle, either - since they've gone so long with the assumption baked in that the problem isn't really solvable.

In other HN posts I've complained about people pointing out things in the Python packaging ecosystem that aren't really problems. But this really is one.

https://zahlman.github.io/posts/2025/02/28/python-packaging-...

Even if your builders are downloading dependencies on the fly, you can and should force that through an artifact repository (e.g. artifactory) you control. They shouldn't need arbitrary outbound Internet access. The builder needs a token injected with read-only pull permissions for a write-through cache and push permissions to the path it is currently building for. The only thing it needs to talk to is the artifactory instance.

you use a language where you have all your deps local to the repo? ie go vendor?

you can always limit said network access to npm.

You can't since a large number of npm post-install scripts also make random arbitrary network calls.

This includes things like downloading and compiling pre-compiled binaries for the native architecture hosted on random servers.

npm is really cool.

npm is a nightmare.

There are plenty of worlds that take security more seriously and practice defense in depth. Your response could use a little less hubris and a more genuinely inquisitive tone. Looks like others have already chimed in here but to respond to your (what feels like sarcasm) questions:

- You can have a submission process that accepts a package or downloads dependencies, and then passes it to another machine that is on an isolated network for code execution / build which then returns the built package and logs to the network facing machine for consumption.

Now sure if your build machine is still exposing everything on it to the user supplied code (instead of sandboxing the actual npm build/make/etc.. command) you could insert malicious code that zips up the whole filesystem, env vars, etc.. and exfiltrates them through your built app in this case snagging the secrets.

I don't disagree that the secrets on the build machine were the big miss, but I also think designing the build system differently could have helped.

You have to meet your users where they are. Your users are not using nix and bazel, they're using npm and typescript.

If your users are using bazel, it's easy to separate "download" from "build", but if you're meeting your users over here where cows aren't spherical, you can't take security that seriously.

Security doesn't help if all your users leave.

The simple solution would be to check your node-modules folder into source control. Then your build machine wouldn’t need to download anything from anywhere except your repository.

In a world with an internal proxy/mirror for dependencies and no internet access allowed by build systems.

Which is not the world we live in.

Speak for yourself.

s/we/I/

It's called air-gapping, and lots of adults do it.

> The miss here isn't that the build container had network access, it's that the build container both ran untrusted code, and had access to secrets.

If you're providing a build container service then you pretty much have to run untrusted code (the customer's) in the container, yes? So then the problem is really just the bad Firebase config... ?

Isn't it really common for build machines to have outbound internet access? Millions of developers use GitHub Actions for building artifacts and the public runners definitely have outbound internet access

Indeed, you can indeed punch out from an actions runner. Such a thing is probably against GitHub's ToS, but I've heard from my third cousin twice removed that his friend once ssh'ed out from an action to a bastion host, then used port forwarding to get herself a shell on the runner in order to debug a failing build.

So this friend escaped from the ephemeral container VM into the build host which happened to have a private SSH on it that allowed it to connect to a bastion host to... go back to the build host and debug a failed build that should be self-contained inside the container VM which they already had access in the first place by the means of, you know, running a build on it? Interesting.

> probably against GitHub's ToS, but

Why would running code on a github action runner that's built to run code be against ToS?

If it was, I'm sure they'd ban the marketplace extensions that make it absolutely trivial to do this: https://github.com/marketplace/actions/debugging-with-ssh

could have just used https://github.com/mxschmitt/action-tmate

A few decades ago, it was also really common to smoke. Common != good, github actions isn't a true build tool, it's an arbitrary code runtime platform with a few triggers tied to your github.

It is and regardless a few other commenters saying or hinting it isn't...it is. An air gapped build machine wouldn't work for most software built today.

Strange. How do things like Nix work then? The nix builders are network isolated. Most (all?) Gentoo packages can also be built without network access. That seems like it should cover a decent proportion of modern software.

Instances where an air gapped build machine doesn't work are examples of developer laziness, not bothering to properly document dependencies.

Sounds like a problem with modern software build practices to me.

Ya too many people think it's a great idea to raw dog your ci/cd on the net and later get newspaper articles written about the data leak.

The number of packages that is malicious is high enough, then you have typo packages, and packages that get compromised at a later date. Being isolated from the net with proper monitoring gives a huge heads up when your build system suddenly tries to contact some random site/IP.

People don't think it's a great idea. In general, its just too much additional work/process - for very little benefit.

You're far more likely to encounter a security issue from adding/upgrading a dependency than your build process requiring internet access.

Note that without a reverse shell you could still leak the secrets in the built artifact itself.