That is not correct. Specifically, the counterexample is if you are a EU citizen living in an EU country and the actual bits are stored in a datacenter in Texas. GDPR still applies in this case, as long as the website owner accepts EU users. If the website specifically doesn't do business in the EU, things are different.
Since the data originates where you are, the data has crossed a border to get to Texas and is therefore subject to GDPR (extra-territorially), I agree.
If you were in Texas when that data was collected from you, and you returned to the EU, I doubt that GDPR would apply to that data.
The only reason the EU is able to get away with this to a degree is the american governments wish cooperation to a point and our treaties. However, I suspect america will only tolerate so much push from the EU.
If you do actual business in the EU they can block financial transactions, and if you don't do business in the EU, there isn't really any incentive to (ab)use the data GDPR covers?
As a startup though, it's absolutely part of your calculus. If you are worried about GDPR compliance, you can choose to just not launch in the EU (actually that might not be enough, I believe you may need to actively refuse EU users) until after you've proven the concept. Although with California's CCPA, depending on the specifics of your situation, you may as well just carve out some time to deal with compliance.
I believe that is still covered by GDPR, but I'll openly admit that I specifically chose the counterexample in order to stay within my factual memory of the law.
Afaik GDPR covers EU citizens traveling abroad, as well as non-EU citizens in the EU.
Edit: It does not cover EU citizens traveling abroad, see thread below.
https://eur-lex.europa.eu/legal-content/EN/TXT/?qid=15288746...
If you're physically in an EU country then EU GDPR applies to you there; if you're physically in the UK then UK GDPR applies to you there.