Children require a certain amount of time, and are given some degree more by any parent who enjoys parenting and/or has higher standards for parenting than what is required. If someone can't find any time for you because of their children, it isn't that their children have been deemed more important than you, it's that every single thing that they spent an hour a month doing with their children has been deemed more important than you.
I'm not being bitter here; my friends with kids found time for me when the kids were young, and now those kids are teenagers, so there's tons of time. It's just that all importance is, is priority. If you can't prioritize something enough to maintain it at all, it's because it's not important to you. Importance isn't some inner strength of feeling; that's a rationalization of being all talk and no action. If you don't feed something, it dies.
No, he's right. You're no longer important to them. They make their life choices and set their priorities such that you are so unimportant to them that they don't spend any time with you at all. Ask yourself this: If something or someone is important to you, do you make choices such that you stop spending time with them? There's your answer.
And they can say "noo you are still important". They should show it or they are a hypocrite. Emotions that are not acted upon are useless.