You need to be okay with losing X% of employees because they're going to quit for their own reasons, or they're going to be hit by a bus, or want to change jobs internally.

This doesn't mean that you push them out, but rather that you make sure more than one person knows how everything works.