Edit: If we're going to use the logic that religious views should be protected due to their inherent "stickiness" relating to family and culture, and other aspects out of one's immediate control, I'm not convinced that other ideological views are any less "sticky" and thus not equally worthy of protection.
You can be a communist without anyone knowing. I have a set of some very odd beliefs, nobody really knows unless they ask.
You can't quite do that with religion when you have days you can't eat meat or you can't work because you have to go to a special building or what not. Or often religion instructs you to talk about it. Ideology can stay in your head; religions (properly followed) tend to be disruptive to one's entire lifestyle, so if a society doesn't protect it, it can as well declare the religion invalid for what that's going to do to it.
Which might be what the society wants to do, but you should understand what it is that you're doing.
Religions are often very sticky by location or by cultural subset, so if you decide to discriminate against religions, often you're not really discriminating against someone's free thought, but just where they happened to be born and what kind of culture they grew up with.
> at least in a free society
Lots of religious people do not come from societies one would reasonably call "free".