You have to consider the fact that it's really weird for everyone to know that evidence against you exists, but be forced to pretend it doesn't and hide it from the jury. It's not at all self-evident that this is the only, or the best possible, remedy against the government abusively collecting this evidence from you. Ultimately it's very important for the good functioning of society that guilty parties are appropriately punished and innocent parties are exonerated, and disregarding evidence can harm both goals. It's of course also important that the government doesn't seize your property or violate your privacy without a reasonable cause, but this doesn't necessarily conflict with the primary goal of justice.

Also, the good faith exception is supposed to be relatively weak - it's supposed, at least originally, to only apply in cases where the officers performing the illegal action had no reasonable way of knowing that it is illegal; the original case is quite clear - a search was conducted in one state based on an out-of-state warrant, and that warrant itself was later deemed to have been improperly issued. I find it quite reasonable to say that the officers conducting the search had no reasonable way of knowing that this warrant was problematic. If the good faith doctrine was watered down so much that an "oops, I didn't know I was breaking the law" from an officer is enough, then the problem lies with these standards, not with the principle.