Right. No one wants the cameras to follow Kirk as he gets a worried look on his face, sticks a copy of Space Sports Illustrated under his arm and runs for the head after eating a bad batch of Rigelian Sarklon worms. Presumably, however, it does happen from time to time.
I think you're right, though. It's not so much that there's no time wasting, but the show was about the adventures of the Enterprise and its crew. For all we know the crew in TOS spent 99% of the time screwing around, and the show was about the 1% of the time they spent working.
Which goes to show--really, the reason they don't show anyone goofing off on Star Trek is because goofing off isn't very entertaining to watch and because it's an action-adventure show about action and adventure, and not a show about people's everyday lives. At least by DS9 they realized it would be entertaining to show the crew trying to get into each other's pants.
It's even full of people gambling, and the gambling tables are run by busty women in cleavagey outfits. And in the first few seasons, it's pretty obvious that the holosuites are for porn.
Quarks Bar? Those aren't business meetings. Those holodeck/holosuite episodes? They weren't there for training. Picard is always reading or trying to read a book, Sisco does baseball and Janeway paints and whatnot with Da Vinci.
Its just that it is also a TV series. It's just not interesting watching Keiko tweet that O'Brien's favorite food is horribly unhealthy and she hates eating it.
Keeping the show interesting is basically the most important thing for the writers, so its probably best not too look too deeply at what is portrayed as if it is some realistic situation.