But this would be after the collective bargaining and signing a contract laying out the terms for union employees. If non-union employees can get better terms, the union can simply annul the contract and force a re-negotiation where they can't be denied because labor law? Or do you mean they could demand re-negotiations and strike otherwise?

Germany is said to be a lot more pro-labor than the US, but as far as I'm aware, unions can't strike during the duration of their collective bargaining contracts.