I think HN likely has a better algorithm for this than Stack Overflow (where I answered about 4 simple HTML questions 12 years ago, logged out, stopped using the site and years later realised I had racked up thousands in karma), but even so longevity is still a huge factor. I'm sure some very active new users do rack up karma quickly, but for most just having an account here for a decent amount of time is what does most of it (and being reasonably active during that time).

I'm coming up on 10 years here... I've never been a big receiver of upvotes, it's just time.