Also, the container itself usually contains (or has access to) valuable secrets, such as keys for staging servers and, of course, the source code.

What I do is that I do all dev work in a VM, usually dedicated to a group of related projects. There are no passwords stored in the VM and its ssh key has read-only access to repositories. The VM has write access to forks and code is merged using pull requests just like third party contributions. This setup has been working well for me, and I even created a tool to automate setting up these dev VMs which I hope to make publicly available at some point.

I don't do any heavy computations except for running test suites of various kinds, and these seem to perform the same on raw hardware as they do in the Kernel-based VMs.

Love to see a container environment that can monitor Monitor and log all outgoing network connection requests.... Monitor and log all critical file/directory access such as /etc/*

With such container, we can catch the compromised supply-chain attach easily, right?

Does anyone know such container exist?

Only using privileged containers, or else you don’t have visibility into signal from other containers.

But, say you had such a container, there’s an important distinction between “you captured a log showing the smoking gun evidence of the supply chain attack”, and “you successfully picked that log out of all of the log data you generated and classified it with high confidence as an attack”.

Speaking from experience, the second problem is the hard problem for a multitude of reasons. So while you would have the data, you’d probably have trouble getting good precision/recall on when to actually sound the alarms vs. when it’s some SRE who needed to troubleshoot some network connectivity issues.

> Only using privileged containers, or else you don’t have visibility into signal from other containers.

The suspect application doesn't need the privileges, so I'm not sure how much of a problem that is?

> there’s an important distinction between “you captured a log showing the smoking gun evidence of the supply chain attack”, and “you successfully picked that log out of all of the log data you generated and classified it with high confidence as an attack”.

Assuming that you're talking about the signal:noise problem, that's hard in the general case but I feel like you could easily pick off really obvious cases like trying to access private SSH/GPG keys and still get a lot of value.

> Assuming that you're talking about the signal:noise problem, that's hard in the general case but I feel like you could easily pick off really obvious cases like trying to access private SSH/GPG keys and still get a lot of value.

Probably. I’d agree that it’s worth trying at the very least. I’ve run into enough “should be easy” cases that turn out to be not that easy that my default is to get the data and see if the hypothesis really pans out.

I’ve created Packj sandbox [1] for “safe installation” of PyPI/NPM/Rubygems packages

1. https://github.com/ossillate-inc/packj

It DOES NOT require a VM/Container; uses strace. It shows you a preview of file system changes that installation will make and can also block arbitrary network communication during installation (uses an allow-list).

strace uses ptrace, which is not safe for security use because of race conditions. Linux Security Modules should be used.

https://stackoverflow.com/a/4421762/711380

Thanks for highlighting this! While PTRACE introduces TOCTTOU vulnerabilities, Packj sandboxes fixes that by using read-only args for ptrace. You can find my PhD work [1] on this relevant.

1. https://lwn.net/Articles/803890/

If CI/ CD pipeline uses GitHub Actions, you can monitor and even block outbound network calls at the DNS and network level using Harden Runner (https://github.com/step-security/harden-runner). It can also detect overwrite of files in the working directory. Harden Runner would have caught this dependency confusion and similar attacks due to a call to the attacker endpoint.

Jails / sandboxes such as bwrap could be enough in this case by denying access to e.g. HOME files not explicitly whitelisted.

Also a Little Snitch-like host-based firewall, which would request explicit permission to connect.

This is the way things should be done by any competent developer.

Generally a VM jail is preferable - firecracker or cloud-hypervisor(virtiofs & gpu passthrough) recommended.

A proper namespace jail (eg. bwrap) is sufficient for 99.9% of cases. To break out of a properly configured namespace jail you would need to sacrifice a 0day.

It looks like totally overengineered solution to me. The CPU already has a protected mode which doesn't allow the program to access any files directly. Why do you need to run a VM which by the way is run from the kernel (privileged mode) in Linux? Why cannot you run untrusted programs in protected mode?

With containers the entire surface area of the kernel is available to attack (syscalls). With a VM the surface is restricted to the VMM and KVM.

This is an oversimplification, there may be other protocols that are passed through or utilized, they would add to the surface.