Well to a large extent the civil service is meant to "run the country". The political leadership is meant to set the rules and directions, but they don't do the day to day grunt-work, and the civil service provides the continuity between governments and a lot of what they do indeed does span different governments without much interference from the political leadership.

So his "joke" might have been a sinister indication that they are sidestepping political control, or just pointing out the realities of who does the actual work.