You can pay for safety explicitly with regulation or implicitly with poisoned children. Regulation hits small businesses harder; rather than concluding that regulation sucks, maybe we should try something else like providing some publicly-funded office to provide compliance help to small businesses.
That doesn't mean it's impossible to start a business, but it does mean you'll get a lot less of them. A lot less of them also means less successful ones and less jobs.
Regulations may or may not be necessary for certain things, but to fold everything in the market under some kind of regulatory framework where you need to go to government to get approval and navigate a burdensome bureacratic process basically just ensures that you will harm innovation, destroy small business, and protect legacy companies in the space.
A great example recently in the United States was in the Consumer Product Safety Improvement Act passed in 2008. It was in response to large toy makers using lead paint in their toys coming from China. It wasn't small toy makers doing this, it was the Mattels. As a result of what these large companies did with their disregard to product safety, a regulatory safety framework was put into place that Mattel could easily absorb into their operating costs, while small mom and pop makers suddenly had a very expensive process to go through, even if they were not the cause of how this law came into effect.
We can all agree on respecting privacy, toy safety, etc. It's a good thing. But just remember that usually these things are passed to protect large companies, not necessarily for the benefit of the consumer, and definitely not for young competitive companies trying to break into a market space that now has a huge initial cost that may be insurmountable. The result sold to the consumer is normally just a side effect used to promote it.