I think the point you brought up about the observance of younger children of their superiors is very true. I can recall fond moments of watching my father work on our early DOS machine. I used to play Doom II and Duke Nukem on Sunday nights as a kid and seemed to learn DOS quickly. However, as I stated earlier, it wasn't until later in high school that my interest seriously grew.
Secondly, there are expectations about it. For example, if a boy takes apart a computer, everyone goes "such a boy", but if a girl does it, people say "hmm, that's odd." (even if it's good surprise, it's still surprise). I know when I was a child I was pushed to do things like ballet and cheerleading and orchestra when what I wanted to do was martial arts and read and become a scientist.
http://www.amazon.com/Unlocking-Clubhouse-Computing-Jane-Mar...
They did a survey of CS students to find out their first experiences. They provide specific quotes in the book, but the gist of it was that men usually were given a computer (or everyone felt that the family computer was "theirs") and they began to tinker around immediately, whether that be with programming or taking it apart. The women's first experiences tended to involve watching a male family member work on the computer.
byoung2's comments follows the typical male answer and mine follows the typical female answer. I watched my 2 uncles and uncle-in-law in the computer since they had jobs in tech. Then my grandfather gave my mom and me a computer and showed us how to use it. I mostly just played on it and used programs until high school were I was "formally introduced" to programming.