I do, to an extent, agree. But I also think it's impolite to have relationships where "we're going to talk about what I want to talk about, but when we go to talk about what you want to talk about I'm just going to pick up my laptop and ignore you or tell you I don't want to".

You could argue, in your description, the same about RMS - he might feel entitled to someone's time to talk about free software.

There is no relationship if you're never going to see these people again. My time is my most precious resource, and I'm not obligated to give it away because someone thinks their initiation of a conversation needs to be reciprocated.

RMS set boundaries and that's perfectly reasonable.