> when the "right" person shows up, how do we know they're the right person?

That prompts an investigation. The “right” person casts an affidavit ballot and the police and courts investigate. If the count is close, the loser usually sure to recount and verify, and any of these incidents then become political kindling. It doesn’t happen because it isn’t worth it individually and difficult to coördinate en masse.

As someone who has lived outside of United States, I find it incredibly baffling, alongside the lack of national ID. Lack of such simple verification makes the potential investigations much more harder than they have to be.

It's a trade-off that many USA states make willingly. Citizens have the right to vote, period^. It's not a "right to vote but only if you have an ID." Requiring an ID to vote, to me, is as ridiculous as requiring an ID to speak or practice a religion.

[^] except for the case of felony disenfranchisement laws, which I personally believe are a travesty

And this was hard won. US history is riddled with examples where the bureaucracy of voting was explicitly used to disenfranchise rightful voters by governmental officials that wanted to keep their power over the marginalized. The skepticism is earned.

> Lack of such simple verification makes the potential investigations much more harder than they have to be

This can be argued for any hindrance to bureaucracy. On the balance you get a much more robust system, with fewer centralized fail-safes.