If you're non-resident in Canada, you only have to pay tax in Canada on Canadian-source income. Unfortunately, yes that includes a pension.

To be non-resident you have to spend at least half the year outside the country and have no residential ties to Canada like property, dependents, bank accounts, health insurance, etc.

See: http://www.cra-arc.gc.ca/tx/nnrsdnts/ndvdls/nnrs-eng.html