The stuff gets brittle over time. Had a windjacket for bicycling where this happened after about a year of usage.

Then, one day coming home, out of the rain, pulling it off I wondered what all those tiny white flakes on my shirt were?

Didn't even realise it until I discovered it was that white inner lining :-)

Wiki says household dust is indeed one of the sources...

All of the above. The coating wears off over time. It can be absorbed through the skin or breathed in as dust.

Is there, any info on the magnitude of these effects? As in wearing a North Face jacket versus eating from a pizza box every couple of months?

From what I've seen there isn't enough research to know for certain. There is correlation with the issues mentioned in the article but I don't think any research has confirmed the pathway for negative effects or the doses necessary to cause them.