You can have socially conservative views. You cross a line when you actively engage in oppressing other people because of your views.