I don't understand why you say leap seconds have nothing to do with wall clock time. Don't wall clocks include leap seconds?

Wall clocks just track UTC. Today they have leap seconds because we defined UTC to include leap seconds, tomorrow if UTC or a replacement universal time no longer has leap seconds then wall clocks won't have leap seconds.

But then people will notice a few decades later that noon in UTC-minus-leap-seconds is not noon.

But it _is_ noon.

You are probably thinking of sun transit time, "solar noon", the moment in each day when the sun appears to be "highest" in the sky. This varies of course by position on the Earth, it's how people set "noon" when they didn't need to agree with anybody more than a horse ride's distance away what the time was. So, let's say at least 200 years or more ago.

Do you know when solar noon is where you live now? No? Because it's irrelevant. Huge numbers of people live in places where the solar noon changes by an entire hour twice a year for no sensible reason. Does this cause a huge problem? No, there's a slightly elevated rate of road accidents and things like that, but nothing major. A few seconds per decade is _nothing_.

What's noon? Depends on who defines it. This is the problem with time.