Knuth is a professor - so in addition to coding, he had to teach classes, advise students, apply for grants (and write the Art of Computer Programming too!). Wolfram has to run a 700-person company. Peter Norvig, Lars Bak, and Rob Pike all spent a long time as managers in giant companies - so lots of time wasted in meetings, etc.

Like you, I enjoy programming a lot, and I consider myself lucky that I get to spend most of my time on it without these distractions. It's rare to be in a field where you can do what you love without being forced to take on the other stuff over time.