RMS can be called an extremist because he refuses to accept that the opposing views have any merit when they clearly do (see some of my other comments on this post for examples, my apologies for hanging around this thread so long haha).
Just because the guy isn't holding college auditoriums full of undergrads hostage in exchange for GNU/Linux being installed on all campus computers doesn't mean he isn't an extremist. The FOSS movement, while stirring up a fair amount of debate and heated views, doesn't lend itself very well to suicide bombing, hostage taking, or any other type of violence. Maybe a little protesting but that's all. Even so, it doesn't mean he's not an extremist.
It's all about that fine line between strongly held belief and extremism. Peaceful protest doesn't equal not being an extremist.
Please don't pull out dictionary definitions of extremism to counter my point. I'm not saying black is white and white is red, I think I'm being pretty reasonable here. There's also a lot of people on the "well, x idea used to be considered extreme" bandwagon. I'd say to you that different isn't always better. Communism, socialism, feudalism, and others used to be considered extreme... And different too. Turns out all the naysayers were right! Those things unfortunately were tried and they all bombed out with everyone involved surely wishing they'd never been tried to start with. I'm not saying, at least not in this sentence and the next, that Stallman is wrong. What I'm saying is consider the possibility - really think about it. If you can do that then at least people like me can't call you am extremist.