The original owner doesn't lose valid title just because it was stolen, and two people can't hold valid title at the same time (unless shared obviously, but you can't have two 100% owners). The second purchaser's title would not be valid, even if they initially thought it was at the time of purchase.
It doesn't matter that you didn't know it was stolen - it's still not yours, it belongs to the person it was stolen from. It's your job to get your money back from the last person, and their job to get their money back from the previous person - all the way back to the thief.
If it's not possible to recover the money from the thief - wow, sucks to be you. You have a claim against the thief, but that doesn't matter if they don't have any money. You can garnish their wages from their McJob when they get out of prison.
Also notable here is that German law, and many other European countries is opposite the English common law rule, and effectively the burden goes the other way and the scope of good faith is widened. Article 932(2) of the German Civil Code covers this for instance.
When dealing with stolen property from 50+ years ago there are issues with the statute of limitations, and with laches as well
Morally, it's debatable. Legally, it's straightforward, a stolen item does not become un-stolen with time, nor does the claim of the original owner diminish. Morally that's not how people think, but legally that's how it works.
The British Museum argument does not work for plebians. It's not "yours" just because you kept it for 100 years. Nor is it the next person's because they didn't know it was stolen.
The British Museum only works for the British because they were a god-ordained royal family that controlled an empire spanning the globe, then a nuclear-armed imperial power. It was based on the rather debatable legal theory of "try and stop us". It's increasingly recognized that it's unjust and seems increasingly likely that it will be dissolved sometime in the next hundred years. Because yeah, the same arguments apply to shit the British looted as shit the Nazis looted. It's not theirs. Maybe some countries will be willing to loan them back on a permanent basis though.