The worldview holding the modern copyright system to be the moral reality is the elaborate rationalization, ingrained into people by the interested parties. You can research the history of copyright law if you want.
This is a separate issue from caring about compensating and nurturing the artists, which is often what people who are mindful about this stuff do. Just not necessarily inside the "traditional" framework.
But there isn't anything immoral about benefiting without paying. At least under consequentialist utilitarianism, one of the main doctrines in ethics.
And why would there be? Someone benefits but no one else loses. If someone has planted a try by the road and I relax in the shade for a few minutes, have I done anything wrong? If I admire a fine bit of architecture have I done something wrong? If I pause to watch the kids playing in the playground have I done something wrong?
You've got it backwards. The onus is on you to provide a convincing argument that sharing documents without paying directly the authors is immoral.
If the document is useful and popular, then I see no immorality in sharing it without paying the author. For paper books, the readers do not pay the authors directly either, they pay the distributor the agreed price. In other cases, this agreed price is the marginal cost of file download.
There's a sort of virtue in embracing the intellectual honesty of saying that you took that work because you wanted it and had the means. Call it the Genghis Kahn justification.
Even if the work is completely derivative, someone still put significant effort into compiling and organizing it. And their immorality of taking the work of others doesn't obviously justify the immorality of taking theirs.