More often than not, one's morning or afternoon is late in the evening or in the middle of the night for the other person. So morning is almost unambiguously referring to the westernmost person and afternoon to the easternmost.

Instead if the working day begins at 23:00 (as would be the case for Pacific time) for me and 8:00 for you, but the meeting is already at 1:00/10:00, it is unclear if you mean "at 1:00 of Wednesday, which is on the working day that started Tuesday at 23:00" or "at 1:00 of Tuesday, on the working day that started Monday at 23:00".