What you’re implying is that considering the circumstances of life is ad hominem, which I do not agree with.
I’d argue that you cannot truly understand the philosophy without the context, it’s baked in. So you need to know the context to truly disagree with the correct thing, but yes the disagreement should be specifically with the idea, not with the person.
Eg not “Nietzsche had a privileged life therefore he was wrong” more like “Nietzsche’s approach to and thoughts on the will to power only really apply when considering the monomania spurred by his father’s early demise and his deep interest in philology. A 21st century approach to nietzschean thought would doubtlessly look very different since material, literary and spiritual conditions have changed somewhat”