I think private contracts would be sufficient. The author doesn't yield his work to a publisher without terms; for the publisher to pretend it is in the public domain (e.g. to start distributing copies without royalties to the author) would violate a contract. The same could apply to end-users; without any explicit copyright protection, you could be required to sign a contract as a condition of purchasing a book, agreeing not to reprint/redistribute. I don't see that the essential parts of copyright protection add anything fundamental; enforcing IP can be reduced to upholding a contract. It's not artificial.
In the absence of copyright the author would have no protection what so ever and would be at mercy of society at large. With copyright they gain protection from the public domain which we, the public, give them in exchange for the cultural creations this protection entices.
If they in exchange of this protection returns nothing to society (i.e. infinite retro-active copyright extensions), then we as a society should have no reason to provide this protection for them.
Basically Disney was able to build an empire on works found in the public domain, but they refuse to give the same opportunity for others to do the same. That is also very one-sided and society as a whole gains very little cultural capital and is forced to pay (again and again) for what they have generously provided protection for.
Make no mistake of it: If copyright law is one-sided, it is to the creators' and media-conglomerates benefit and not society.
Society isn't owed anything. Government (which is distinct from society) protects IP, not because they benefit from it or society maximizes utility from it, but because that is what justice is about. This isn't a free service; the authors pay taxes for (among others) the legal system that protects their rights and enforces their contracts.