Compare Newtonian causality where there is only one timeframe everywhere to SR causality with its local timeframes to GR which supports warping of space and time. All different and not interchangable.
Wormholes cannot exist in SR, therefore applying SR's brand of causality is equally invalid. Under GR the concept of simultaneity is relaxed: the effect is only limited to the light cone of the cause.
I agree that if the space Between A and B is distorted enough so that the light that travels in open space between A and B is seriously affected by a curvature that extends from A to B, then it is true that you cannot use SR to make your predictions, as the premises of SR are violated. But then GR will predict that causality can be violated anyway. (It's generally easier to find ways to violate causality in GR that SR anyway, since it doesn't require FTL transmissions of information in GR to create causal loops.)
I refer you to the FTL faq, since it seems that I am not able to talk sense into people on my own:
If, on the other hand, your FTL drive bends up space all around us to be like a giant hyperspace pretzel, (1) we're going to have a lot more problems than just a little violation of causality, and (2) causality still ends up being violated via General Relativity.