I think the term "IP" is meant to highlight the one big similarity they share. We recognize that this thing, which is more of an idea than an object, belongs to an entity (person/company/government/whatever), and that entity has control over some of the finer details of how that thing is used. That's even pretty readily apparent in the term itself: "intellectual property", property that is more of a thought than a thing. Maybe it's a bit euphemistic, and to someone with Stallman's beliefs, the propagation of the term probably seems more than a bit dystopian, but it's still a useful umbrella.
Can you elaborate on those similarities? Stallman, in his essay[1], states some important differences. He makes a coherent argument. So what similarities here are "rightly" recognised? I'm not saying you're wrong, but "nobody's confused" is not really a complete counter-argument.
[1] https://www.gnu.org/philosophy/not-ipr.en.html , viz,
"These laws originated separately, evolved differently, cover different activities, have different rules, and raise different public policy issues.
For instance, copyright law was designed to promote authorship and art, and covers the details of expression of a work. Patent law was intended to promote the publication of useful ideas, at the price of giving the one who publishes an idea a temporary monopoly over it—a price that may be worth paying in some fields and not in others."