Had the same :) But it is not really true. You can get a french bank account without living in France, got mine while living in Germany. But of course not all people working for a bank know or respect that. We got checked by my consellier very carefully before she said oui to opening the account.
Nice is the other one: To get an appartment you normally have to have an insurance, but to get that you need an appartment. Works with the landlord accepting to get the assurance attestation one week later, but is goodwill. Of course, a couple from marokko I know did not get the key from the man responsible in such a situation, while getting the appartment because it was university housing. They stood on the street after arriving here… That was pure racism, but shows that many thing here need goodwill to work despite the system.
It's more of an issue today since so many people just use their debit cards whereas years ago people used cash and credit cards. If you wanted the convenience of not carrying cash, you had to use your credit card since ATM cards didn't have Mastercard/Visa logos and couldn't be used to pay for things in stores. That's how today you wind up with someone in their late 20s that's never had a credit card or a loan.
This was me till I was 28. I was able to skip having to use a secured credit card by getting a joint card with someone who already has great credit.
Not specific to France - I had the same experience as a French in Berlin (though it was twenty years ago - it might have changed).
Sweden was the worst on that aspect. Most owners require that you have a company signing the lease on your behalf (supposedly to guarantee the rent, yet they generally refuse advance payments on the rent).
And if you're required to supply some documentation from another country, you're likely to be required to get it translated to French by an official translator (which not only takes time but can be quite expensive). In contrast, I've never had a problem handing in German documents to authorities in Sweden and Swedish documents to authorities in Austria. Documents in English have certainly been accepted everywhere outside of France.
The catch-22s are the best. Renting an apartment requires that you have a French bank account, and getting a French bank account requires that you already have somewhere to live in France...