I think another part is how much diversity there is in speakers of the language and the common dialects.
For example, there's a very large number of Spanish speakers worldwide with a very wide variety of accents and dialects, plus large numbers of non-native speakers. That also applies to English, but doesn't apply so much to French (not that there's zero variety, but compare the content of https://en.wikipedia.org/wiki/Spanish_dialects_and_varieties, https://en.wikipedia.org/wiki/List_of_dialects_of_English and https://en.wikipedia.org/wiki/Varieties_of_French).
Because of all that linguistic variety, if you're an English or Spanish speaker you're generally more accustomed to hearing different pronunciations of your own language day to day than French speakers are (on average, YMMV, etc).
I lived in Romania at one point, where this was even more obvious. There are comparatively few distinct dialects of Romanian, and most Romanians have never heard non-native speakers speaking their language, so no matter how friendly they were, understanding unusual (OK, wrong) pronunciation was just not a common skill.
Keyword: refuse. They can, but pretend they don’t. It’s racism disguising itself as cultural pride. (Source: I’m French and am certainly not proud of that part of my heritage)
It's elitism, France's hate of foreigners is not founded on skin colour or heritage (e.g. both your parents can be French and if your French is bad, you get treated poorly anyway).
But yes, elitism is also present: If you're perceived as French and your french is bad, you will also be treated poorly. But it's a different kind of "poorly". In fact, Paul Taylor talks about that in his stand up "Franglais" (I HIGHLY recommend a watch, it's hilarious especially if you're familiar with both French and British/American culture, and it's available for free on Youtube: https://www.youtube.com/watch?v=Pae2AMnmUVA)
My heart wasn't really in it, but I was also very discouraged by this.
I think it's a mix of multiple reasons.
First, French is a language which not only has no word stress but actively blends words together when speacking in a sentence. Correct word pronunciation is key to understanding and what foreigners view as small errors sometimes do actually make their sentence unintelligible to a French speaker from France.
Then, language proficiency is a strong social marker in France. So, if you speak broken French, people will subconsciously view you as lower class/uneducated.
Meanwhile, Spanish, with its perfect phonetic spelling is spoken by a people who will go out of their way to understand you no matter how bad you speak their language.
My experience is that it’s mostly cultural, and often comes down to whether the person you’re speaking to has themself had to try using a foreign language, thus gaining the empathy to help you out.