If you have e.g. public health system, and public education, at high standards, your chance of success is to a vastly greater degree decoupled from the wealth of your family, or the suburb you live in.
The benefit is that society as a whole gets richer, you reduce crime, etc etc. Basically what comes around goes around.
To be opposed to having great public benefits e.g. just because you are from a family of great wealth and are unwilling to share some of that for the greater good is thus something I ultimately define as selfish. I also think it is short sighted, as no matter how well off you are the higher the chances you'll be affected by things like crime.
Hopefully this explains my thinking.
Maybe libraries are a moot point, given the Internet. But the same issues come up for schools, police, fire, roads, bridges, etc. I mean, I have no children, so paying for schools seems like a rip-off sometimes. But on the other hand, I like living in a society that's at least somewhat literate.
I was surprised to hear you describe people voting against continued public service as 'selfish'. Why would you consider them selfish?