Right, barring time travel, once you can make the voyage at all, you'll always reach a point where it makes sense to depart on it. If you invent technology that takes a million years to arrive, then you have a maximum deadline of a million years. Past that point, even instantaneous travel won't be worth waiting for, if your goal is simply to get there as early as possible.