What I had in mind is, you can observe your planet's position and velocity (relative e.g. to center/rotation of the galaxy), and the target planet's relative position and velocity, and use both to translate the observations into virtual "stationary" observations... And "unit of time" would be defined as 1/n of one planetary/galactic revolution, so every observer would be able to calculate their own "galactic time" independently. Well, maybe GPS already does something similar.
If you take a reference with you on a journey through the solar system, that reference will stay accurate for an observer travelling with it and drift as seen from an observer that stayed behind. Both views are accurate. Locally, the amount of time that has passed for each observer was different. If you would correct for any observed drift, you would mess with your fundamental unit of time in your local frame of reference and get incorrect measures of time passed in your own frame.