Also, at the OS level, there are entire projects built around just understanding the dependencies in the Operating System and applying minimalism to try and get to a state of (relatively) simple, known dependencies.

One of my favorites is Rob Landley's Aboriginal Linux:

http://landley.net/aboriginal/about.html

I highly recommend his blogs if you're interested.