It can be hard. It may be less painful to do it slowly and gradually over time rather than some sudden pronouncement.

To some degree, any time anyone says they "can't" do something like this, they are saying they don't want it bad enough. I have done a lot of rearranging of my life to get things to work in a satisfactory fashion. This includes but is not limited to changing my relationship to how I use my phone. I used to be a big phone person. I now do most stuff online. After a long time of doing very little with my phone, I am slowly doing more with it these days.

But there is also this: If your family, friends and work all are basically making you crazy, they would probably do so even if phones did not exist. So, this might be an indicator of deeper problems.

Or it might be an indicator that you aren't being driven as crazy as other people, so it just isn't really that important to you.

For most, I think there are easy ways to massively reduce time spent on their phone without an adverse effect on career or relations otherwise, but that's predicated on a want to. Besides, I see plenty of people who are on their phone all the time, yet their life is almost void of physical interaction with others. I think it's very common to overestimate the importance of being "on", and even more common to think it has a positive effect on their lives.