The primary definition (OED) is "forming an exception", which is why I think this cliche is a tautology ("exceptions are for forming an exception") that does nothing to guide me on whether an exception is appropriate in a given case.

Btw, we don't say "exceptions are for infrequent conditions", because that's not what they're for.

I quite like the etymological "taken out", because it carries a notion of special handling - a control flow aspect that is the main point of using them.