If you're from a big country like France or Germany, you don't notice, but if you're from a smaller country like Portugal this means when you realise your country does not even have consular facilities in the random place you're visiting, you don't worry, the French or Germans do and you're an EU citizen, so they'll sub for your own consular officials as needed.
Free travel is not the feature that is legally indicative of a nation.
[1] Yes, I know that EU consulates can be shared by other EU citizens. Nonetheless, legally there is no such thing as "an EU consulate" or "an ambassador from the EU" or a generic EU passport. It really is a radically different model from the EU. (The most notable difference is that countries can unilaterally withdraw from the EU without triggering a civil war within the EU. States can't.)
Every single EU member state is a sovereign nation with its own foreign policy, military, standing to sign treaties, seat in the UN General Assembly, international recognition beyond the UN, and a couple of them even have permanent seats in the Security Council. And most, but not all are parties to the Schengan Area, and most, but not all, are members of NATO. The ones that are not landlocked even get to set their own maritime boundaries, whereas past a certain point in the US, the maritime boundaries here are Federal territory. EU member states still maintain formal diplomatic relations among themselves and with other nations, and issue passports and visas. They might have standardized their passports, sure, but they are still issued by the member states.
Hawaii has none of that, it is a US State. Once you’re a US State, you don’t stop being one, ever, and we fought a civil war over this. That clearly isn’t the case with the EU, as demonstrated by Brexit.
In short, the analogy doesn’t work because it isn’t analogous. The EU wants to be treated like the US in some cases and like separate nations in other cases. The US is just the US as far as other nations have to be concerned, and there isn’t a Hawaiian or Californian or Texan passport that other nations can granularly accept or reject, just American ones.
If you want something more analogous to the Japanese hypothetically rejecting Hawaiians, it would be like the US allowing Visa-free entry from all of Germany except people from Brandenburg, or all of Australia except people from Victoria, or all of Japan except people from Okinawa. As far as the 4 EU States without Visa-free access to the US are concerned, they have normal diplomatic channels available to them.
EDIT: Edited for grammar and clarity. Sorry about that, quite a few interruptions in the last few minutes.
If you find that difficult, just rephrase "the EU has" into "the EU member states have". So: the EU member states have decided on collective visa measures.
Collective response is something states do. See NATO Article 5.
It isn’t a mere organizing principle, there is a legal distinction that is easily drawn with differing legal trappings.
The EU deciding on collective Visa measures is fine, I actually have not taken any issue with that in this entire thread, the only thing I took issue with was a bad analogy which left alone, only confuses and misleads otherwise constructive discussion.
Long and the short of it is: Hawaii and Croatia aren’t analogous to each other vis a vis visa agreements, and that’s because they come from differing International legal positions, a position in which Hawaii is actually irrelevant from beginning to end and Croatia isn’t but has the legal standing and recognition to temporarily be irrelevant in its own visa policies if it chooses to. The nuances are not unimportant.
Case in point: I have an EU driver's license. It is valid in the EU. When in the US, driver's licenses are per-state, though I am allowed to drive in a different state for a while, I have to get the current state's license when I move my residence. California has border checks for agricultural goods. etc. etc.
Nations have agreed to fairly consistent diplomatic protocols and channels with which to treat each other. Hawaii can’t issue visas, but Japan can require Americans to apply for a visa. That isn’t just a stretch of the Commerce Clause, it is part of how and why the US Constitution was written the way it was: so individual States weren’t forming international interests contrary to the national interest.