So you’re asking why bringing up that he used to be perceived as and treated as a woman which gave him gender dysphoria — a feeling so bad that it causes 80% of trans people to seriously consider taking their own like and 40% of people to actually try — horrible?
You being up a good point about memories. But memories are mutable, first person, and we’re really bad at remembering how our bodies felt at different times so most of his memories probably feature him as he is now.
According to your ideology, he never was a woman.
But... he was. He was a baby girl, a teenage girl, an adult woman, all or at least most of that whilst I assume suffering dysphoria. He is now an adult man who, unfortunately, likely still doesn't feel "complete".
Grandpa suffered trauma during the war. So we don't bring up the war. We don't pretend that he never served. You can be sensitive about history without rewriting it, we do it for everything else.
Society really should not have to pretend they were never a woman just to protect a person's feelings. And why is acknowledging that they used to be a woman be so horrible? They have their own memories of their life before transitioning.