In English it's all translated to "tax" and rightfully so. English languages and cultures don't have subtle semantics like French does. For example everything in the UK is a tax, corporation tax, income tax, council tax, etc... (one exception being the UK national insurance).
Social Security is a called an "insurance program" although it is completely unrelated to conventional insurance. The closest financial instrument is a Ponzi scheme, which is illegal under US law.
Health coverage goes to some entity that isn't the government and is usually optional, so it's viewed as a benefit and taken into account when computing total compensation.
Unemployment insurance on the other hand is taken out of every paycheck and handed to the government without an opt-out. Yes, of course you get something for it; the whole point of taxes is that society gets something in return. They're still taxes.
With respect to claims of wanting to underpay employees, I don't think that's something anyone is actually arguing for when they talk about unfriendliness toward startups. The French government is levying a high _proportional_ tax to cover _fixed_ fees like health care. This can make sense at a societal level as a form of wealth redistribution, but it's a disadvantage for high earners like those in the software industry. Those employees can earn more in other countries with the same cost to their employers simply because less is taken out of their paycheck, and because the difference is more than enough to cover fixed fees like health care it's a good move for them to do so. The people being disadvantaged aren't startup employees (not passing a moral judgement -- the claim itself just seemed off).
And things like tax credits are only useful if you have the cash on hand immediately, and can wait up to a year to get it back. A brand-new venture may not have that.
France has good public health service but it's also pushing private health insurance down the throat of everybody through their employers, quite like the UK or the US. The large majority of the population has double coverage.
They include social security, unemployment benefits, retirement benefits, training ...
When your employee is sick, they can stay at home without losing their pay, and without your company paying for it. They can seek medical treatment without risking bankruptcy.
If you have to lower you activity because business is bad, you can put in place part time unemployment, and your employees won't lose money in the process...
Also, the fees paid by the employer are often subsidised through eg tax credits, reimbursements (up to a certain amount)... So that these numbers are not only inaccurate but also totally meaningless.