Morally, it's debatable. Legally, it's straightforward, a stolen item does not become un-stolen with time, nor does the claim of the original owner diminish. Morally that's not how people think, but legally that's how it works.
The British Museum argument does not work for plebians. It's not "yours" just because you kept it for 100 years. Nor is it the next person's because they didn't know it was stolen.
The British Museum only works for the British because they were a god-ordained royal family that controlled an empire spanning the globe, then a nuclear-armed imperial power. It was based on the rather debatable legal theory of "try and stop us". It's increasingly recognized that it's unjust and seems increasingly likely that it will be dissolved sometime in the next hundred years. Because yeah, the same arguments apply to shit the British looted as shit the Nazis looted. It's not theirs. Maybe some countries will be willing to loan them back on a permanent basis though.