No, it doesn't matter. Where I live, the time the sun is at its highest varies from 11:53 to 13:18 and at no time in the history of my city (I checked all the records) has a child ever asked why the sun isn't directly overhead precisely at noon.

Well, you must not live where I grew up, because I asked this very question...

...which lead to me grokking time zones, time, solar noon, etc in a way I don’t think I would have if I hadn’t asked. Somehow solar noon not being noon didn’t have a negative impact on me as a kid. Weird, right?

My answer was (hopefully obviously) tongue-in-cheek with regards to this never actually happening, but more importantly, it was a response to the idea that children would find it strange that the sun isn't directly overhead at noon. I don't know how a child could find that strange if it so rarely even happens that it is directly overhead at noon.

What? Is there any place in the world that distinguishes "noon" and 12:00pm in everyday speech?

You're asking for a lot more trouble if you want to separate them.

How about "afternoon" -- can that start at 11:00 am?

(Yes, scientists can use the term "solar noon," but I'm talking about everyday speech.)