If you are contracting for multiple clients, you should probably be cool. The thing which is problematic is if you only work for a single client (the mentioned "Scheinselbstständigkeit" - your claim to be self employed, but you really aren't). In this case Germany really just wants that you are regularly employed by that client company. (the reason is that this was commonly abused to force cheap labour into self employments, so that the employers also don't have to pay gov insurances and work around a whole lot of other laws ...)
As for the company being incorporated in what will become a third country, from my research I can't see any reason why that will fall afoul of any regulations (as long as everything is properly reported to the authorities).
I've head rumours of German contractors using UK limited companies instead of the local GmbH/UG entities. How does that work generally? Are people setting themselves up as German employees of the UK entity and collecting a salary? Or is there something I'm missing here?