The problem is actually that people are forced off of their homes, personal property and way if life should there be a big accident. For an average householder it's their life savings, friendships, community and way of life wiped out through no fault of theirs. Sure there'll be compensation and resettlement but who trusts governments to do this fairly and painlessly?

It hasn't helped that no civilian nuclear operator has demonstrated the ability to sustain safe operations with zero incidents and consistently prioritize safety over the course of decades. Every operator has a string of nuclear incidents of varying severity and i think more than anything, the Fukushima thing shook people up because while you could excuse away Russian and American accidents with cultural factors nobody perceives Japanese to be careless or irresponsible (and this was expressly the reasoning that led Angela Merkel to reverse her stance)

So yeah in abstract at a nation level, it's not a major risk but for the families possibly affected, it's a catastrophe - as opposed to increased chance of a few people getting cancer over their lifetime from Coal.

Therefore given the asymmetric risk and demonstrated inability of the nuclear industry in ensuring zero incidents, there's naturally grassroots opposition which translates into political pressure that no amount of "risk is so low" data-flashing can wave away.