Even if it were though, it seems obvious being on the receiving of that slur would have significantly less impact on you, as it didn't actually target anything you saw as part of your identity. I don't see how your experience puts you in a position to absolve others for their use of the term.
Personally, I think we should give significantly more weight to folks who are actually in the impacted community (those with intellectual disabilities, their loved ones, etc). The vast majority of whom _do_ object to the use of the word as a derogatory slur.
The parent comment already covered this. You don't believe you are, based on how you've talked about this experience. At no point do you say you identify with the word, just that it was used against you.
> If we trust the experts on this I'm severely handicapped.
I mean they themselves basically said they do not identify with it:
> If people ran around calling you the N word on a daily basis would it be a comfort to you that you aren't actually black?
The other kids who called you retard are not experts in this particular diagnosis.