I cancelled my participation in a free software conference this year, but I doubt it will be much noticed except by the core team (I help with internationalization). I also didn't make too much noise about it, not to harm the event/community. Most of our European partners already do not bother going to the US and organise their own conference.

You may notice is less and less diversity, but it's already pretty low, and we're often not very good at noticing that.