Hmm, not so sure I'd trust "safe enough" in a flood plain, earthquake zone or valleys around a volcano.
Some people won't be able to afford the difference, and every so often you'll get a news story about 1,000-100,000 dying in a natural disaster.
On the other hand, houses do tend to be safe enough even without regulatory oversight. I definitely think that the years that people in developed countries spend working and saving in pursuit of an expensive house built to code is not a good balance of life spent to life risk.