Well, not sure I follow you on this terrain... Let's find something that we would agree is useless, and enjoyable. Maybe fishing alone and putting fish back in the river?
Or you would say that this is also useful. But then anything is useful and the word itself has no meaning (and become useless, hehe).
I may be too much influenced by Eastern streams of thoughts, where we have zen, taoism, etc., all saying that nothingness, emptiness, uselessness are things we should value more, because they are the ground on which everything grows. (Simple example: without silence on which a composer is building, no music is possible)
So, to go back to pg's essay, which is more of a short blog post, I found it was lacking an open door toward these possibilities that "lost time" is not always lost.
The short essay isn't really about declaring activities to either be productive or wasted. It's about learning to identify "empty" activities that waste time with the false promise of being productive. Whether your time is wasted or productive is a personal matter for you to weigh against your own goals and values.
P.S. My apologies to anyone who might take offence to my poor understanding of Zen.
My impression is that you are reducing utilitarianism to a Mr Spock kind of pseudo-logical thinking. You seem to be forgetting that the ultimate goal of utilitarianism is happiness. It's not about gaining knowledge or even self-improvement. Playing cards for pure fun is not at odds with utilitarianism.
(Disclaimer: I am not a philosopher nor do I have more than a basic knowledge of utilitarianism)