Containers have made the conceptual shift from allow/deny to isolate/share. Somehow this feels better even though it is effecitvely the same.
I am still waiting for an abstraction that uses all the low level features and wraps them in a high-level interface that puts usability front and center.
I am not sure if this is even possible though because many applications are not built with sandboxing in mind. Adding another file somewhere on the system that needs to be accessed is not considered a breaking change by most. So maybe we need a more fundamental shift.
Bubblewrap? FlatPak's sandboxing features are built on it already.
Maybe that's just daydreaming about a perfect world, but I agree that it should be easy to reason about the access levels.
Vim is exactly the kind of application you would run in a virtual machine.
Also, afaik firejail runs as suid, making any possible escape much more serious.
> For a server, the process exposed to the outside world runs as an unprivileged user (unbound or nobody). The process is started by a separate process running as root (as explained by @Ferroin above). The starting process is never exposed to outside.
> The same is true for Firejail. By the time the unprivileged server process starts, Firejail is already sleeping.
And I think Docker has a similar problem as mentioned in the "warning" section in [2]:
> Warning: Anyone added to the docker group is root equivalent because they can use the docker run --privileged command to start containers with root privileges. For more information see [3] and [4].
We've had container breakout CVEs in the Kernel+ContainerD+CRI-O this year, and the ContainerD/CRI-O ones were not complex at all, so you do need to stay on top of patching as well as the other good practices mentioned.
Definitely the CRI-O one (which is most likely to affect Red Hat based systems) was able to exploit a default OpenShift cluster in my testing.
1) The article author is Testing in PROD
2) selinux debugging relies on auditd, so sanity checks required.
df -P /var/log/audit # has space?
tail -1 /var/log/audit/audit.log # is recent?
semodule -DB # disable dontaudit
setenforce 0
# run the failing test
audit2allow -l
mk_semod() {
module_name=$1; shift
audit2allow -l -m ${module_name} -a > ${module_name}.te
$EDITOR ${module_name}.te || return
checkmodule -M -m -o ${module_name}.mod ${module_name}.te
semodule_package -o ${module_name}.pp -m ${module_name}.mod
semodule -i ${module_name}.pp
}SELinux also supports interesting things like applying security levels and contexts to data, so you can have data that is only accessible to appropriatly cleared users in one department but not to people with similar clearances in other departments. Very few people use this stuff though.
Selinux uses extended attributes to store its labels which is a pretty standard way of associating metadata with objects, and is supported by most standard unix commands (via the -Z flag) so I'm not sure what you mean by "filesystem hacks". Fundamentally they are both implemented as security modules in the kernel and do pretty much the same thing, which is best depends on whether the added security of SELinux is worth it to you.
No, AppArmor uses a whitelist approach as well for its profiles. "Deny" rules are used in exceptional cases.
> SELinux also supports interesting things like applying security levels and contexts to data, so you can have data that is only accessible to appropriatly cleared users in one department but not to people with similar clearances in other departments.
AppArmor has subprofiles and variables that can be used for this approach.
I was daemonizing a uvicorn (Python) process using systemd and could not get the Python script to write its log file anywhere. Not the current directory owned by the user, not elsewhere, and no errors showing up in the system logs.
After a few hours banging my head against the wall I discovered systemd's ReadWritePaths. Once I set that, it would log to any directory I specified.
It's the only service I've created where this has been necessary, and others write logs, so I have no idea why this one caused errors. If I did something wrong, let me know how this should be handled.
It sounds like you were using the DynamicUser=, ProtectSystem=, or ProtectHome= settings which can greatly limit where a service is allowed to read from and write to and thus require you to open specific directories back up. Instead of using ReadWritePaths=, you might prefer to set LogsDirectory=.
Thank you! I have DynamicUser set - I can't think why I added it here, perhaps no reason at all - and that is the difference to previous systemd services I've written. I did not know it had such an effect on behavior.
The DynamicUser= setting tells systemd to invent a new user on the spot, with an id that is not in use by any other service. You can add as many such services as you like¹ without having to do any extra work to keep /etc/passwd updated.
¹ as many as you like as long is it isn’t more than several thousand, unless you recompile systemd with larger limits.
Maybe systemd/journald supports this use case by forcing you to adopt some sort of structured log format that you emit to stdout, but I don't think I want to have to conform to whatever journald wants me to do, unless it's standardized and well-supported among various logging tools.
Try `journalctl -o json` to see all the fields that get logged. Applications can add more.
So it very much means that a webserver logging to journald can already provide it all with fields for the vhost, status, URL, referer, and whatnot, and you can just search the log by that and don't need to do your own parsing.
I'm just tired of all this added complexity, when I don't really see much of the benefit.
> I'm just tired of all this added complexity, when I don't really see much of the benefit.
There is a lot less for each application to implement on its own, such as log levels, timestamp format, log rotation or even just the paths where logs should go. Instead those details can be configured by the system administrator using a uniform interface across all applications.
What a great time to be a Linux system administrator.
Meanwhile, I am holding my own just fine, latest systemd-wise.
https://GitHub.com/egberts/easy-admin/blob/main/500-dns/512-...
It is all about reading the friggin’ source code … of systemd.
edit: I like how I'm getting down-voted for sharing factual information. No-one in high frequency trading employs containers because it doesn't help soft-realtime, low-latency goals. There are other performance-sensitive domains where containers aren't an option. Sometimes I feel like this place is turning into Reddit.
What cases can linux containers not handle? Containers can access GPUs, /dev/kvm, block devices... I'm having trouble thinking of anything they can't do. After all, they're just processes in a glorified chroot, not that different from processes on the linux host.
https://elixir.bootlin.com/linux/latest/source/include/linux...
Also docker NAT will add significant latency if you don’t use host network mode.
When microseconds matter, everything matters. Any intermediate program would be suspect.
"Container" (namespace to be precise) is not really an intermediate program, it's a state associated with any Linux process.
A container is just namespaces and cgroups. You can still have your mellanox or solarflare zero-copy network stack for a containerised process.
That is, a containerised user space process can directly access the hardware, bypassing the kernel but isolated from other processes.
There are plenty of footguns for sure though but my experience working in HFT has been everyone is pretty elite skill-wise so sharp edges are not an issue in practice.
Now that's an interesting problem to have!
If you trust those tools and don't have untrusted users or untrusted code, you can sometimes just mount /var/run/docker.sock and use the VM/VPS/server's Docker directly. It is actually the approach that was used by excellent tools like Portainer, though it's also really risky as well.
Alternatively, you can try to just run Docker in Docker (DinD), which is a bit more tricky and the opinions there are split about whether to do it and when to do it: https://github.com/jpetazzo/dind
Of course, someone might also jump in and suggest that Docker is architecturally problematic (i don't care much, just want my containers to run, then again; i don't deal with untrusted code or any sort of multitenancy) and you should use Podman or another set of technologies, which is interesting advice but would necessitate other approaches.
In short, like with most technologies: Docker and OCI container in general get more messy as your requirements become more advanced. For the problems that they do solve easily (app packaging), they are pretty good, though!
It's currently working great and I have not had any major crashes/incidents for at least the past 8 months.
If it's advisable is another question, as that gives full control over all other containers (and likely more)
At that point you may want to just run it on the host, but basically you'd better trust whatever it is.
In the k8s env I run, the filesystem is backed by ceph, which creates persistent volumes that the different pods can claim. Is it just "be careful not to mount docker.sock"?
Absolutely not. They are not designed as security tools in the first place. Especially docker, with its huge attack surface.
You are confusing containers with sandboxes.
Furthermore they don't implement anything resembling RBAC.
Or just use Qubes OS: https://qubes-os.org.
With IllumOS I can agree on zones, I can partially agree on lpar on AiX, I can accept GNU/Linux limits with cgroups. Enlarging the attack surface to follow the current wannabe Gurgle mania... No. Like no in the recent past to full-stack virtualization on x86 to enrich VMWare, buy big desktops sold as powerful servers just to play matryoshka doll for someone else business...
Of course, it's hard to satisfy hw "subdivision" vs sw subdivision, in the past we have tried the hw path with blades for the small side, infiniband and co for the large side and both proved to be not very scalable but the issue is in actual OS design and can only be solved changed that: An OS mush be a single application, with internal namespacing, networking. We have seen it in various past OSes. It's about time to make them again accepting that the IBM model is good only for IBM, witch means for selling stuff to people having no choice but buy or ignoring something better can be done.
Aside from purely guessing I'm interested in scalability pains war stories with infiniband too :-) as I'm looking into a 400GbE-everywhere datacenter build...
In classic times that's just a matter of balancing both, carefully designing both, but in modern time developers just "produce code", operation is pushed aside, communications are constrained often by the policies and the result it's a mess => companies choose "the cloud" witch means outsource the issue to third parties. Those third parties count on the fact that even if anything change in the end a computer is always a computer so they can live on a changing world because if single companies have irregularly changing needs on average the rate of change is similar. Also on scale redundancy needs lower the costs and that's why we switch from classic big iron to desktop crap packed in racks. Now we start seeing more and more the limit of such model, and no solution is here.
Big iron is long lost, now way to expensive not much in absolute terms but in demand terms: try to sell a Power system, the customer look at specs and say "hu?! a couple of PCs from nearby grocery store are equally powerful!". Such iron modularity coming from classic mainframes (infiniband in the end is crylink changed a bit by SGI and others) is considered too expensive, on the other side of the spectrum small-board ARM super-blades tentative still results in fails.
Until we accept that hw and sw can't really be two separate world and both must be open to being able to interoperate on scale we will not find any really scalable solution.
Sw side it's not different: these days is the containers mania days because after many have suffered burn with full-stack virtualization on x86 they look for something with less overhead and equal design: hw-sw decoupled, possibility of run an infra you do not know buy pre-made images/containers from third parties who maintain them without needing to know your infra. The result again is an enormous amount of layers and giant attack surface no one really know how extended it is. Since coms are needed we punch holes here and there, and on the other side "safety features" arrive to compensate, making more complicatedness layers on complicatedness instead of admitting the basic issue: in sw terms our OSes are badly designed for current needs, they are a relic form a long lost past where they aren't even good but just cheaper alternatives to something better.
If we ever admit those errors probably we will be able to design modular iron and modern OSes marginalizing those issue, but so far no one want such a nightmarish adventure...
I would say it's not necessarily easier to manage, but it's much easier to understand, because containers create natural borders between the different software running on the machine.
And if you enables uid mapping, your permission in the host system is just as high as nobody if there isn't a kernel exploit happened somewhere.
But the fact it is nobody is also problematic, because dir mount will be almost alwyas read-only now.
That is, it does nothing to address security problems latent to each container binary.
Now, I think my advice would be: Put everything in a container, and learn how to run Docker or Podman (or k8s) in a secure way (ie no root containers, be very careful with volume mounts, etc). Yes, they aren’t as mature as SELinux, but containers aim to provide many of the same benefits that SELinux does (and even more) except in a way that’s much easier to manage. Even better is that these container runtimes often come with SELinux and AppArmor policies out of the box on good Linux distros.