There’s a lot of background thinking that happens with knowledge work. Spending 3 hours at a desk and 5 hours milling about in the office isn’t necessarily “not being productive”. It’s pretty often that I’ll get stuck on a problem, distract myself on HN for a few minutes, then pump out a solution in minutes or otherwise have an epiphany.

Similarly, talking to coworkers, even (especially) at unrelated parts of a company can be super valuable, even if it codes as “unproductive time wasting”.

TLDR your definition of work might be too constrained.