Doing business in the EU is only one path that causes the GDPR apply to you, processing personal data of people located within the EU, not necessarily EU citizens, is another one and probably the one relevant here.

Article 3(2): This Regulation applies to the processing of personal data of data subjects who are in the Union by a controller or processor not established in the Union, where the processing activities are related to: (a) the offering of goods or services, irrespective of whether a payment of the data subject is required, to such data subjects in the Union; or (b) the monitoring of their behaviour as far as their behaviour takes place within the Union.

But what's the worst that could happen? Your site would get dns blocked in the EU?

Maybe they could arrest employees on a visit to the EU? Maybe you could even get arrested at home due to extradition agreements? Or get assets in the EU frozen or seized? But that is all pure speculation, I have no idea what actually can and can not be done or what is likely to happen.

They are perhaps owned by media organisations that do in some way do business with the EU even if those specific outlets do not.

as soon as some of some assets pass through entities in the EU (or even, entities on wich the EU can apply pressure), they would be seized.

It's still unknown how exactly they will enforce the GDPR, but they've got data on Europeans thus need to comply. The US does this quite often as well having their laws applied to foreign companies, without them being physically present in the US.