I have no idea how we'll survive that.
Or an alternative(maybe): all technologies will be inside highly regulated environments.building a 3d printer from scratch is really hard.
It definitely is right now. Suppose for a moment that a 3D printer were invented that could turn raw materials into perfect copies of itself. Suppose also that the raw materials could be automatically extracted from household and electronic waste. A machine that can turn garbage into copies of itself and other objects with similar complexity to the original machine.
The only remaining obstacle would be software (and data files). If we are to assume that machine learning and AI progress in parallel to the development of such a machine, it's difficult to see how we could ever prevent somebody from putting the pieces together.
Then we developed nuclear weapons, and everyone finally realized the end was in sight, no need to panic, the threat was finally existential and real. Our response to this realization was to mutually stockpile enough nuclear weapons to destroy most human life and all human civilization many times over. Still, like the crossbow nuclear weapons have not totally destabilized society, and now intellectual people are looking for the next threat, and for the first time all of them are potentially world ending. That doesn't mean the world will end, as we've seen many times in the past, but as you say the gap is narrowing, and we're never going back to a time and we don't have the means to actually end it all.
I think people worry about robots because it's the most comfortable thing to worry about, compared to nukes and climate change at least.