Well, there /is/ another way to do it.
STATIC LINK ALL THE THINGS
Which would work if licenses and copyrights didn't exist.
We're at the point in the hype cycle where it starts getting fashionable to dismiss that as an overkill, but the reality for most of us out there is that most software is way more complicated than a single executable and containers make it easier to deploy complicated software.
Managing internal dependencies (like libraries) is another concern entirely. But containers are good for that, too.
I don't think it would.
Dynamic linking allows a library to be patched once and have the patch apply to all the programs using it. If every program was statically linked, you would have to update each one individually.
Not to mention the waste of space.
I'm guessing much of that is moot these days, but IMHO it's still something to aim for.
The benefit of that goes away with containers anyway, you don't share libraries, every instance gets its own install.
Modern package management systems like APT spend a lot of effort installing and removing files, and they don't do it completely; any file created by a program after it is installed will not be tracked.
You could accomplish the same thing in other ways (as Apple's sandboxing tech does), of course.