Ironically, I keep getting told I have to switch over from docker because of security. Plenty of IT security folk have their knickers in a twist at the idea of non-sysadmin priviledged users. They would rather run a giant insecure system (Linux or Windows AD) with lots of LPE and lateral movement than smaller more isolated systems with more trusted users and less LPE. Dev(Sec)Ops concepts still haven't penetrated very deep.
There is quite a bit of distrust of Singularity places that value security. I'm hoping podman rootless can eventually come through and render all the bespoke implementations moot, but it will probably be a few years.
While it would be better to run a capability micro-kernel-ish system, you're stuck with Linux for things like HPC. There's no excuse for known exploitable privilege escalations, though and, yes, introducing more opportunity with setuid programs doesn't help. I agree the way such systems are run is unfortunate.
Singularity makes some bad design choices though, like eliminating layers, which results in proliferation of huge container images cluttering everywhere, with no idea what they do. It's really designed for HPC administrators who don't like end users.
https://sylabs.io/guides/3.5/user-guide/security.html