The original poster means: given that you have paid all your taxes, etc, the marginal cost for you of going to the library and taking out a book is 0.

Of course the infrastructure costs money to maintain, and the people working there are paid.

The phrase "free at the point of service" is sometimes used to describe this

Whenever someone says X is free, they usually mean: X has already been paid for via other means (taxes) and doesn’t have a usage fee. E.g. free beer doesn’t just come out of the ground (and if it did, you would still need to spend resources on extracting it).