But the current system, with all of the "security" it provides, already has a handful of really really bad people that commit crimes. So why should I give up freedoms to try and stop these bad people when despite everything, crime still happens?
I'm not sure what you argue exactly. That policing has no effect on crime rates?
The modern state we have is here not to build Heaven on Earth, but to avoid Hell. Eliminating all crime is impossible, but without state intervention you end up with Somalia, Afghanistan or post-Katrina New Orleans.
You don't need "majority", you need just a handful of really really bad ones.