Yet somehow everytime I see these fairly dramatic predictions about the US losing its petrodollar and the dollar collapsing or banking moving to $other_country, $cryptocurrency, etc or everyone switching to the Yuan or Ruble, etc they are almost always wrong.

I think you're dismissing how powerful inertia is in the financial sector and how the EU needs the UK and as such will work out agreements to keep things going. London wealth and banking is most likely very safe. Remember, London as the premier financial center of Europe predates the EU by a few hundred years. It is not an outgrowth of EU membership.

London is still the banking center of Europe. Trade cuts both ways too: the EU needs the companies and people based in London. Cutting themselves off from their own financial center won't be pain free.