Why that is controversial, I have absolutely no idea.
I generally agree, which is why I tend to have a very high bar for calling someone a Nazi.
However, "fascist" is a rung below that. I have generally taken the definition to mean "ideologically selective disrespect for [democracy/rule of law/both] motivated by both a claim of victimhood and a belief in the superiority of [some ethnic/cultural heritage], where particularly the claim of victimhood involves some kind of secret manipulation on the part of [political activists/foreigners/minorities]".
It is unfortunate that a major political party in my country has come so close to satisfying this definition ever since the Ukraine affair (confirmed extensively by John Bolton in his recent book), which I spent some time studying/developing around 2016 mostly to ensure that I would be intellectually consistent in determining whether there were actually fascists in the US government. It has never been with eagerness that I called anyone a fascist.
He didn't call you a Nazi. What he said was: you're alleging the misuse of a term, but you're doing so while being very vague and giving brief, angry replies to comments all over the place, and this... pattern... is characteristic of bad-faith and/or extremist commenters on threads about anything posted almost anywhere.
If you have some information you want to talk about in detail, I suggest you do so; it is much nicer to carry out discussions when we don't have to guess what the point is.