It depends on the country and the financial institution. I'm a permanent resident of another country and had no problem opening an account there(on vacation there the first time no less). Another user here reports a very easy experience opening an account with an institution in the Netherlands.