Sure, there is some paperwork to set everything up. But once that's done (and you can pay someone to do it for you), it's pretty much smooth sailing.
The biggest "hassle" is closing the accounts for the year and sending them to the Administration, and those have to be signed off by an actual accountant. But this takes like 1 hour per year, it's not that bad.
Regarding health insurance, etc you can either get your own, private one or have it bought by the company. It's the same thing if you're a full-on employee: if you're not happy with the coverage the employer has, you can get your own on top of it. When you're your own boss, you can choose what you want straight away. All this may vary depending on the type of company you set up (there's no such thing as "freelance" as far as the law is concerned). This will also have an effect on pension – if you only pay yourself in dividends, there's no pension.
Sure, there are a few things to be taken care of, but if you invest a few days and do your research, it's a good tradeoff.
The other approach, as siblings said, is to use some kind of "shell company" that does this for you. But from what I've seen, their cut is ridiculous, so it's absolutely not worth it.
NB: this comes from France, but I expect the situation to be roughly similar in other countries.
Then the company you own pays all these things. Not the company that hired my own company. It's more a B2B relationship than employee mentality.
I hired another startup to do all my accounting since it's really simple.
I keep some cash inside my company to pay taxes and so on. In my jurisdiction it's even cheaper the tax you pay as company than as an individual.
I have some sort of salary because my local legislation demands, but most is taken as dividends.
The spare money due to reduced costs I can also afford extra private "unemployment" insurance. One that covers two months, not even counting my personal emergency reserve.
So yes, you need to stop living by paycheck even if it means buying less stuff on Amazon.com.
Sounds like a massive headache