If we consider the combo of exceptions+RTTI, then yes, dropping both at the same time lets you get rid of some further overhead (although again it's memory-only, not compute cycles). But then we have to discuss the lost benefits of those other features, not just exceptions alone, to judge whether doing so is worthwhile or not.
BTW, I may well be wrong here, but isn't RTTI generation for exceptions (when it's otherwise disabled) an example of "only pay for what you use"? i.e. wouldn't the compiler then only generate RTTI entries for those types that are either thrown somewhere, or are caught somewhere? If so, any library code that doesn't throw itself, and only needs to allow exceptions to pass through it, is still zero-overhead in that regard.
I don't really see this as much of an issue given you have to have this RTTI to catch the exception properly.