I have the exact same experience, though I starting learning English at a young age. Out of the 100+ Brazilians I met in Business School, I was the only one who was interested in hanging out with non-Brazilians.

There's a reason I left Brazil, and it mostly has to do with wanting to live in a better culture.