Not all British accents carry that stereotype.
I've personally experienced situations where people seem to think I cannot understand some simple concept when I am not a native speaker. Just an an anecdote, though
It happened to me often when interacting with Chinese CS professors during college. I'd constantly have to remind myself that they were much more knowledgeable than I was despite having subpar communication skills.
Which British accent? It's unlikely that you associate prestige and sophistication with scouse, cockney, geordie, scots, west country, so on and so forth. These are generally played up as being working class or not smart in movies and other media.
Oi boyz, we gotz errselves a posh git roight 'ere.
'Respectable businessman' Badger from the Firefly universe probably wishes that were true for all 'British' accents!
That's too generalized. To an American, a British accent conveys prestige and sophistication. This has been studied, not just my opinion, though I can't immediately find a citation.
Also, to me, certain foreign accents sound very charming. Arnold's voice sounds authoritative, but it's probably more to do with the deep pitch than the accent.