Western countries have in fact denied visas to foreigners who publicly hold "inappropriate" views. Visiting a foreign country is considered a privilege not a right, so it's not something where there's much if any concern about supposed "rights".