I believe you are wrong. If the license doesn't apply as written, then nothing in copyright law gives you any rights. GNU has always argued that if you don't accept the GPL, then your use of the software is pirating it under copyright law. That is the same consequence as would happen if I sold a library for money, but instead of buying from me you copied it.

If you want to be evil and have written any GPL you can sue everybody in the target country that uses your software. Since the GPL isn't legal in their country residents of that country have no legal right to use your code. Note that you need to write "mass market software" for this to apply.

Of course I'm not a lawyer, consult a lawyer if you need legal advice.