I agree with that view, but I suspect this issue comes up a lot because many people don't agree with it. A common view is that private property is a natural right of some sort, merely recognized by society, rather than a convention created by society.
If you took that view, then whether intellectual property is legit turns largely on whether it is or isn't part of this natural-law category of "property" (which is what you find debate about in libertarian writings on IP, for example).
My point is not that the question at stake is straightforward, but that once we stop clinging to flawed analogies, the matter reveals itself as a policy issue like any other, an issue that is complex but ultimately resolvable by the usual means of analysis.
All the rest is rhetoric.
You can consider protection of private property the most important right in societies like ours, so important it occupies a lofty stratum of its own, and yet not commit the fallacy of claiming that private property is a natural right. However, it's understandable why people with priorities like that would make such a mistake.
This is why "infringement is theft" arguments are so manipulative and dishonest. They try to tap into this indignation rather than win the debate on the merits.
Rights should be judged by their usefulness, not by how fundamental some person thinks they are, because no right is truly fundamental.
Interesting use of the passive voice - who's doing the judging?
My point is that there are two views of property: it's a right that is somehow inherent/moral/pre-existing; or it's a convention that exists for its benefits to society. How we can know whether it has benefits, and who decides whether the convention should exist, is a whole different discussion.
I'd say that if we're going to view 'society' in a meaningful way - that is, not merely as a conceptual abstraction, but as something exists empirically - then we have to acknowledge that society has a definite nature, emergent from human psychology, emotions, physiology, neurology, etc.
And if we accept that society exists, why should we not recognize that the recognition of things like 'rights' and 'property' emerge from the very same antecedents? These concepts are all descriptions of fundamental relationships between people and other people, and between people and their physical environment.
The idea that social conventions exist for the benefit of society is something of a tautology. Obviously, anything that anyone creates or does is something they're doing because they've judged that it will be of benefit to them. The ideas of 'law' and 'government' follow the same rule - these are also just conventions that people established, in the expectation that they will somehow benefit.
But the real question, the empirical question, is this: what are the actual motivations that have led people to establish these conventions? I suspect that the idea of law emerged after the idea of property, as people sought to maintain the relationships they established with physical objects in the context of their relationships with other human beings, and sought a structured and formal way to do so without violence (or perhaps merely dissension and group harmony, given that cooperation is as essential to our nature as competition).
Fast-forwarding to today, with all of that in mind, I think it's reasonable to make a distinction between the conventions that arose from a 'natural' social context and those conventions which are the product of positive law, the latter being a kind of 'secondary' experimental thing, intentionally designed by the mechanisms of the earlier first-order convention.
In my mind, 'theft' is a violation of the first kind of convention, while 'copyright infringement' is a violation of the latter.
It is important in a debate to not allow people to vilify any participants in "piracy" by declaring them to be criminals from the get go. "Theft" does this. "Copyright infringement" does not. Definitions are important because without sharing a vocabulary, we share nothing with which to debate logically.
Protection of private property is to prevent theft. We both can't have your car. If I copy your software, we both have it.
Protection of intellectual "property" was granted as incentive to create. It was (is?) a time based monopoly on your creation. At some point the protection ended. This was a benefit to society as it allowed ideas to be built upon.
It's completely irrelevant what you call it unless you are one of those curious people who considers private property to be almost an ontological category. Protection of private property exists to the extent it's judged useful to society. Protection of so-called intellectual property exists to the same extent. Counterfactual what-if reasoning is as valid or as suspect in this debate as in any other.