The first is that you have to get to a significant percentage of light speed for this to manifest itself. Even at 0.95c time is only traveling at 1/3 of normal speed [1].
Second, the energy cost for this is truly mind-boggling. Doing so when you carry the fuel seems highly impractical. The best bet are laser highways [2]. But, you need to build that first.
The upper limit of laser propulsion isn't really high enough for that level of time dilation.
Third, traveling at these speeds makes any potential impact within even a speck of dust potentially fatal.
Lastly, fun fact: such a ship actually needs to be aerodynamic. Why? Above 0.95c the drag of hydrogen atoms in the interstellar medium will start to slow you down.
On the other hand a civilisation producing such things should hopefully be technically immortal with lots of distributed backups for each individual potentially affected by such accidents & resources to fix up any affected infrastructure. So still, totally worth it, go fo it! :-)
(Less advanced neighbors might still be a bit on the edge, so please be considerate.)
Okay, maybe you can use a laser or something so you don’t have to carry the fuel with you, but you’ll still need a staggering amount of energy because the rocket equation is a harsh mistress, and the relativistic version is even worse.
---
The book itself is only so so but the concept is interesting non the less.
As long as they don't have relations with normal humans outosde the caste they live quite normal lives, meeting their friends from the caste relativity often from subjective PoV, with potentially decades or centuries going one between such encounters.
Still if you say want to settle down and have children, you won't likely wont see anyone else from the caste ever again as the next time one of your friends ventures to the system you settled in they will only encounter your descendants, possibly a couple generations after your time.