I'm not blind, nor deaf. I have a family and have never been isolated. I just assumed this is How Things Are, that attention must be paid in an effort to stay "in sync."
Granted, I was keeping a vary regular schedule and there where plenty of clocks around which helped, but I think most people can probably get to that level of precision.
Depression is also strongly related to being in the dark, probably linked to us needing vitamin D.
I attributed the difficult to a diagnosis of mild dyslexia and ADHD (if you believe in that sort of thing). Since these diagnoses in my early 20's I've learned these tricks (such as rationalizing event order).
Now, in my early 30's, this has just become habit. I'm still not as good at remembering events as others but with the mental tricks, I'm adequate.
Interestingly, people who have lost their sight or hearing, at birth or from a young age, experience this too.
From my own experience (back when I was deaf), I would (and occasionally still do) perceive relatively large stretches of time (up to a few hours) as considerably less, and vice-versa.
Funnily enough, though understandably I suppose, I also have a very hard time placing events in time in relation to each other, it very often becomes an exercise in rationality (i.e. event A could only have happened if event B had too).
Sadly, I can't find much information about this, except maybe this (which I don't have full access to) http://www.mitpressjournals.org/doi/abs/10.1162/jocn_a_00135...