I'm mildly skeptical that a court would accept this reasoning in cases of works with many copies and someone making a change to one such personal copy.

Well that's a bloody stupid idea.

Is it? Is it that simple?

I write a book about the suffering of a particular minority. The book sells internationally and earns awards. My name becomes recognised globally.

Someone buys many copies of the book, modifies them to say the opposite of my intended point, making me look like, say, a racist, and distributes it.

Now is it stupid?

My point is your name being attached to a particular piece of work means you're name is directly associated with that work and what it represents. Any changes to the work could alter the perception that others have of you, towards the negative, and you've done nothing wrong.

We've recently seen what happens in a post-truth society when someone says something (usually online) that's not true about another person and the negativity generated towards that person happens before the truth is revealed (and promptly ignored anyway.)

Do we really need more of that?

I don’t think we need a concept of intellectual property at all to talk about why fraudulent behavior is wrong/harmful.

You forgot the important, the moral part is only enforced if: “the modification in question does not have to be detrimental to the author's honor or reputation in order to qualify as a violation of the right of integrity.”