Why do we need to make it a software problem? The existing system makes it a trivial mental calculation.
The amount of people complaining here about how difficult it is to keep a handle on illustrates it's not a trivial mental calculation, even for HN. I would agree, especially when coordinating multiple, distant parties.
Apple already do have support for specifying the time zone in appointment times, and the Clock application can show the current time in different places. What woudl you want to add to this?
Attach the current, local time to all contacts, at the basic, fundamental level. If contacts are in a different zone, every app accesses an OS-defined UX, that displays a contacts local time in a ubiquitous, familiar fashion all users get used to.
We have all kinds of low-level protocols for coordinating all manner of things. Distilling a coherent system of agreeing on a timezone difference, between two communicating entities and presenting that to the user at any time would be quite basic.
We have location data for all, at all times. We have a local, accurate clock. There's little else we'd need. What remaining issues are all UX. Apple could roll such a thing out parallel on all their multitude devices and it'd be basically trivial.
Is it maybe just habit that we gravitate to a manual process, when an automated one would be rather easy in this situation?
On iOS, the existing clock app already provides a basic form, tapping the plus button allows the addition of multiple zones and provides useful additional context in the form of offsets and what day it is relative to the users local date and time.
That's halfway there, it would only need contact awareness to be able to provide a solution.