There are lots of things that have artificial scarcity through the simple act of ownership and this is one of them.
The same goes for the "it won't deprive you of anything" argument. There are plenty of things we all own that if someone borrowed it wouldn't necessarily negatively affect us, but that doesn't mean that we should lend them while we're not using them, especially not for free simply because it would satisfy someone else's desires and pocket if it were free.
There should be better delivery of items given the tech we have (it's better than it was but could be much better); there should be copyright laws that benefit the actual creator more, not just huge corporations (these appear to be getting worse); the price of media is often too high (though again, it's a lot better than it was); none of these are solved by pretending we could or should live in a communist utopia.
If you don't believe that intellectual property is legitimate, why isnt it OK to deprive someone of potential income from it in the cases where it is monetized? Does the attempt to monetize it grant some sort of exclusivity?
What makes it okay is that intellectual property can be copied infinitely without depriving the original owner of it, and the fact that they refuse to sell it to you means you're not even depriving them of potential income, so there's no way of claiming there was any damage done.
It ends when you start talking about actual property that has limited physical supply. Intellectual property isn't real. It naturally has infinite supply, so its market value should be zero.