What you say would apply largely to certain parts of right-libertarianism.

As a left-libertarian, the NAP is the kind of sophistry that makes me not consider right libertarian truly libertarian, as taken to the extreme it can be used to actively deprive others of even the right to life, reducing liberty more than it increases it. (Most commonly is it used to protect property rights)

E.g. if I monopolise all sources of food, strict adherence to the NAP prevents you from using violence to prevent everyone else from starving to death.

The left-libertarian view would be that the right of life is far more essential to liberty than property rights - if you are dead there is no way you can be free -, and so if many are at the threat of starvation over your property rights, liberty is maximised by taking your property from you, even at the cost of your life if necessary, as your act of monopolising an essential resources is morally equivalent to violence against everyone deprived.

This is a common good argument and individual liberty argument at the same time.