In my case, I'm not white myself, so there's no way my question would be perceived as "so why are you different from us?".
If you look at my family tree, I come from Fujian, Guangdong, Taiwan, and Canada. We suspect there's also some undocumented Japan in there too.
This isn't trying to split hairs - there's legitimate reason to recognize all of the above in my heritage. My "origin" is complicated and is really hard to pin down into a single word.
So you end up with what stands out most strongly in my head - I'm Canadian, because that's where I've lived the longest, my primary language is that kooky Canadian flavour of English, and the sight of fries, cheese curds, and gravy inspires a craving like nothing else.
While you say you aren't white, do you have any cultural habits that people from White European backgrounds might find abhorrent, disgusting, or icky? Perhaps something that is difficult to explain? Do you want to explain away every cultural practice in a country that you've never been to?
I can think of a million reasons why one would not want to explain or talk about about cultural issues about a country they are originally from (if there is only one country in question). The best way is to avoid letting it become a topic of conversation. This isn't "cowardice", it's avoiding a boring, useless (and potentially offensive) conversation.
May be some do have a sense of self-hatred like you said, but I would guess most people are simply annoyed by those questions so they respond adversely. Like others have mentioned already, it's really frustrating from a point of view of a person who might be a 3rd generation Canadian, as the questions can make him/her feel like perpetual foreigner. Why can't you take "Canadian" as an answer when those people have lived there for all their lives and might have never even visited their grandparents' country of origin?
As you've said you're an immigrant, your ethnicity and the fact that you're a first generation immigrant might be a big part of who you are (if you do treat them importantly, that is). For second generation Canadians and on, that might be not so true. It's not that they are embarrassed by their respective ethnicity, but it just doesn't play a big part in their lives in their thoughts.
I think it is the wording of the question that is likely to bother people. It makes the questioner sound as if he is not aware there are native Canadians who are Asian. And that is a likely reason for why they responded in the manner they did.
It's also a rather normal, conversational, question.
See also 'What do you do?', 'Did you catch the game last night?', 'Have you seen Avatar?', etc.
"Where are you from?" isn't synonymous with "what is your ethnic background?", especially when there are 3rd and 4th generation Canadians of Asian decent.
A lot of people will recognize when being asked that "Where are you from?" is supposed to be a politer way of asking "So why aren't you white?", but refuse to play along because it feels kind of asinine to say you're "from Taiwan" when your family has been in Vancouver for close to a century and you've never even been to Taiwan.